Method and electronic device for easily searching for voice record

ABSTRACT

A method of writing a specific time point through a familiar pattern input that can be instantaneously applied to a portion desired to be memorize or highlighted by the user during audio recording. An electronic device according to an embodiment disclosed in the present disclosure may include a storage unit configured to store audio data and the recording information of the audio data; a controller configured to convert an input audio signal into audio data to store the audio data; a display unit configured to display one or more texts based on the execution of a speech-to-text (STT) for the input audio signal; and an input unit configured to receive a specific pattern input or a selection input for part of the texts from the user while receiving the audio signal.

CROSS-REFERENCE TO RELATED APPLICATION

Pursuant to 35 U.S.C. §119(a), this application claims the benefit ofearlier filing date and right of priority to Korean Application No.10-2012-0053834, filed on May 21, 2012, the contents of which areincorporated by reference herein in their entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method for facilitating the retrievalof recoded voices and an electronic device for implementing the same.

2. Background of the Invention

With the rapid development of the information age, the importance of aninformation input/output function and a data storage function has beenemphasized in an electronic device. The electronic device having thosefunctions can be classified into a portable electronic device such as amobile station and a stationary electronic device such as an imagedisplay device, a refrigerator, and the like.

As the functions become diversified, the mobile station is implementedin the form of a multimedia player having complicated functions such ascapturing still or moving images, playing music or video files, gaming,receiving broadcast signals, and the like. Furthermore, thosecomplicated functions may be also applicable to a stationary electronicdevice for the user's convenience.

In recent years, in order to implement such complicated functions,various new attempts have been applied in the aspect of the hardware orsoftware. As an example, a user interface environment is provided toallow the user to easily and conveniently perform audio recording orretrieve and select audio files.

In general, there is a case where the user wants to memorize orhighlight a specific portion during the process of recording voices.However, the process of retrieving the contents of voice files is notvery intuitive compared to the process of retrieving the contents ofimage files, and thus there may exist difficulties in retrieving therelevant portion again later. Furthermore, the process of taking a memofor a recording time of the portion desired to be memorized orhighlighted and then retrieving that portion may be also burdensome.

SUMMARY OF THE INVENTION

The present disclosure is to provide a method of writing a specific timepoint through a familiar pattern input that can be instantaneouslyapplied to a portion desired to be memorized or highlighted by the userduring audio recording and an electronic device for implementing thesame.

In order to solve the foregoing task, an electronic device according toan embodiment disclosed in the present disclosure may include a storageunit configured to store audio data and the recording information of theaudio data; a controller configured to convert an input audio signalinto audio data to store the audio data; and an input unit configured toreceive a specific pattern input from the user while receiving the audiosignal, wherein the controller reflects a time point at which thespecific pattern is received on the recording information.

On the other hand, in order to solve the foregoing task, a method ofcontrolling the operation of an electronic device according to anembodiment disclosed in the present disclosure may include converting aninput audio signal into audio data to store the audio data; receiving aspecific pattern input from the user while receiving the audio signal;and reflecting a time point at which the specific pattern is received onthe recording information.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this specification, illustrate embodiments of the invention andalong with the description serve to explain the principles of theinvention.

In the drawings:

FIG. 1 is a conceptual view illustrating a representative operation thatcan be implemented according to the present disclosure in a mobileterminal;

FIG. 2 is a flow chart illustrating an operation control process of amobile terminal according to the embodiments disclosed therein;

FIGS. 3A through 3D are views illustrating a first operation controlprocess of a mobile terminal according to a first embodiment disclosedherein;

FIGS. 4A through 4D are views illustrating a second operation controlprocess of a mobile terminal according to a first embodiment disclosedherein;

FIGS. 5A through 5D are views illustrating a third operation controlprocess of a mobile terminal according to a first embodiment disclosedherein;

FIGS. 6A through 6G are views illustrating an operation control processof a mobile terminal according to a second embodiment disclosed herein;

FIGS. 7A through 7E are views illustrating an operation control processof a mobile terminal according to a third embodiment disclosed herein;

FIGS. 8A through 8J are views illustrating an operation control processof a mobile terminal according to a third embodiment disclosed herein;

FIGS. 9A through 9F are views illustrating an operation control processof a mobile terminal according to a fourth embodiment disclosed herein;

FIGS. 10A and 10B are views illustrating an operation control process ofa mobile terminal according to a fourth embodiment disclosed herein;

FIGS. 11A through 11C are views illustrating an operation controlprocess of a mobile terminal according to a fifth embodiment disclosedherein;

FIGS. 12A through 12G are views illustrating an operation controlprocess of a mobile terminal according to a sixth embodiment disclosedherein;

FIGS. 13A through 13C are views illustrating an operation controlprocess of a mobile terminal according to a seventh embodiment disclosedherein;

FIG. 14 is a block diagram illustrating a mobile terminal associatedwith an embodiment of the present disclosure;

FIG. 15 is an internal block diagram illustrating an image displaydevice according to an embodiment of the present disclosure; and

FIG. 16 is a block diagram illustrating a display controller and a mainbody controller contained in a refrigerator when an electronic deviceaccording to an embodiment of the present disclosure is therefrigerator.

DETAILED DESCRIPTION OF THE INVENTION

It should be noted that technological terms used herein are merely usedto describe a specific embodiment, but not to limit the presentinvention. Also, unless particularly defined otherwise, technologicalterms used herein should be construed as a meaning that is generallyunderstood by those having ordinary skill in the art to which theinvention pertains, and should not be construed too broadly or toonarrowly. Furthermore, if technological terms used herein are wrongterms unable to correctly express the concept of the invention, thenthey should be replaced by technological terms that are properlyunderstood by those skilled in the art. In addition, general terms usedin this invention should be construed based on the definition ofdictionary, or the context, and should not be construed too broadly ortoo narrowly.

Incidentally, unless clearly used otherwise, expressions in the singularnumber include a plural meaning. In this application, the terms“comprising” and “including” should not be construed to necessarilyinclude all of the elements or steps disclosed herein, and should beconstrued not to include some of the elements or steps thereof, orshould be construed to further include additional elements or steps.

The suffixes “module” and “unit or portion” for components used hereinmay be merely provided only for facilitation of preparing thisspecification, and thus they are not granted to have a distinctivemeaning or function.

Furthermore, the terms including an ordinal number such as first,second, etc. in the present disclosure can be used to describe variouselements, but the elements should not be limited by those terms. Theterms are used merely for the purpose to distinguish an element from theother element. For example, a first element may be named to a secondelement, and similarly, a second element may be named to a first elementwithout departing from the scope of right of the invention.

Hereinafter, the embodiments disclosed herein will be described indetail with reference to the accompanying drawings, and the same orsimilar elements are designated with the same numeral referencesregardless of the numerals in the drawings and their redundantdescription will be omitted.

In describing the present invention, moreover, the detailed descriptionwill be omitted when a specific description for publicly knowntechnologies to which the invention pertains is judged to obscure thegist of the present invention. Also, it should be noted that theaccompanying drawings are merely illustrated to easily explain theconcept of the invention, and therefore, they should not be construed tolimit the technological concept disclosed herein by the accompanyingdrawings.

A mobile electronic device disclosed herein may include a portablephone, a smart phone, a laptop computer, a digital broadcast mobiledevice, a personal digital assistant (PDA), a portable multimedia player(PMP), a navigation, a slate PC, a tablet PC, an untrabook, and thelike. Hereinafter, various embodiments according to the presentdisclosure is based on a mobile terminal such as a portable phone, asmart phone, and the like, but a function applied to the mobile terminalmay be also applicable to a stationary electronic device such as adigital TV, a desktop computer, a refrigerator, a projector, and thelike, as well as to a portable electronic device that does not use acommunication network such as a PMP or the like.

FIG. 1 is a conceptual view illustrating a representative operation thatcan be implemented according to the present disclosure in a mobileterminal.

The mobile terminal 100 drives a recording application according to auser's request. For the operation, a plurality of applications includingthe recording application may be stored in the storage unit of themobile terminal 100. Referring to FIG. 1, a graphic user interface (GUI)of the recording application is displayed on the display unit 151 bydriving the recording application. Furthermore, the mobile stationrecords an external sound as an audio file using the recordingapplication.

Furthermore, while driving the recording application, the display unit151 may display one or more texts based on the execution of aspeech-to-text (STT) (hereinafter ‘a speech-to-text (STT)-based text’)for an input audio signal. Referring to FIG. 1, the mobile terminal 100is configured to write a specific time point during audio recording. Forexample, a specific time point may be entered by a touch to the text.When the touch selection is sensed, the controller 180 may write aposition (or word or sentence corresponding to this) corresponding tothe text selected from the STT-based text displayed on the display unit151 on the recording information of the audio data.

When an audio file is reproduced subsequent to terminating recording, anindicator indicating the specific time point may be displayed on the GUIof the reproduction application (GUI of the recording application whenreproduced in the recording application).

When a specific time point is written on a recording file through theforegoing process, the user may easily retrieve a marking portion fromthe stored audio file.

A retrieval method of the recorded audio may be associated with a seriesof processes such as recording, storage and reproduction. Accordingly,the series of processes will be divided into embodiments and describedbelow in detail, and the process of retrieving the recorded audio willbe described in detail in one of the foregoing embodiments.

FIG. 2 is a flow chart illustrating the entire operation control processof a mobile terminal according to the embodiments disclosed therein.

The mobile terminal 100 performs the steps of starting a voice memo(S100), progressing the voice memo (S200), and terminating the voicememo (S300). A first embodiment disclosed herein will be described withreference to FIGS. 3A through 5D regarding the step of progressing thevoice memo (S200).

Furthermore, when the vice memo is terminated (S300), the mobileterminal 100 performs the steps of providing a file name (S400) andmodifying the file name (S500). A second embodiment disclosed hereinwill be described with reference to FIGS. 6A through 6G regarding thestep of providing a file name (S400), and with reference to FIGS. 7Athrough 7E regarding the step of modifying the file name (S500).

Furthermore, the mobile terminal 100 performs the step of accessing afile list (S600). Regarding this, a third embodiment disclosed hereinwill be described with reference to FIGS. 8A through 8J.

Furthermore, the mobile terminal 100 performs the steps of retrievingthe file content (S700) and editing the file content (S800). A fourthembodiment disclosed herein will be described with reference to FIGS. 9Athrough 10B regarding the step of editing the file content (S800). Inaddition, a fifth embodiment disclosed herein with reference to FIGS.11A through 11C, a sixth embodiment disclosed herein with reference toFIGS. 12A through 12G, and finally, a seventh embodiment disclosedherein will be described with reference to FIGS. 13A through 13Cregarding the step of retrieving the file content (S700).

In general, an audio recording process is carried out through theprocess of converting an analog voice into a digital form to store it inthe buffer when audio recording is started, and generating an audio fileto store it in the memory using data stored in the buffer when audiorecording is terminated. However, the user may want to control a voicebeing recorded during the recording (and prior to terminating the audiorecording) according to circumstances. A first embodiment which will bedescribed below relates to an operation for controlling a voice beingrecorded while continuously progressing the recording, and the firstembodiment may include various operation examples.

First, a first operation control process will be described withreference to FIGS. 3A through 3D.

FIGS. 3A through 3C are views illustrating a first operation controlprocess of a mobile terminal according to a first embodiment disclosedherein.

The user may be required to check the content of audio recording thathas been made at a previous time point such as several seconds orseveral minutes ago during the audio recording. To this end, the usermay terminate audio recording currently being progressed and reproducethe generated audio file to check his or her desired content. However,terminating audio recording and reproducing the generated audio file andthen restarting recording is burdensome as well as ineffective.

Accordingly, a first operation control process according to a firstembodiment disclosed herein is related to a method of allowing the userto check audio recording corresponding to a previous time point duringthe audio recording in real time without terminating the audiorecording.

FIG. 3A is a flow chart illustrating a process in which the mobileterminal 100 according to a first operation control process disclosedherein reproduces the recorded content during the audio recording, andFIGS. 3B through 3D are conceptual views illustrating a process in whichthe mobile terminal 100 reproduces the recorded content during the audiorecording.

Referring to FIG. 3A, the memory 160 (refer to FIG. 14 below) stores anaudio signal input through the microphone 122 (refer to FIG. 14 below),for example (S1110). The controller 180 (refer to FIG. 14 below)converts the input audio signal into a digital form to store it in apredetermined region of the memory 160, for example, buffer.Furthermore, the controller 180 may store the time information of thestored audio signal. The time information may include information on atime that has been elapsed until the relevant audio signal is receivedor stored after starting recording.

When the mobile terminal 100 converts the input audio signal into atext, the controller 180 may convert the input audio signal into a text,for example, using a speech-to-text (STT) function or algorithm. Thecontroller 180 may store the converted text in the memory 160. In thiscase, a speech-to-text (STT) execution application may be driven inlinkage with a recording application.

During the storage of an audio signal, the display unit 151 (refer toFIG. 14 below) displays an item for indicating a progressive state ofthe audio signal being stored (S1120). The item for indicating aprogressive state may be a progress bar. The progress bar may include anindicator indicating a current time point of the whole recorded portion.When the whole recorded portion is not revealed, the indicatorindicating a current time point is displayed at the middle of theprogress bar. The progress bar may visually display a current recordedtime (for example, 3 minutes) within the whole recorded time (forexample, 5 minutes).

As a more specific example, referring to FIG. 3B(a), the controller 180displays an audio recording screen on the display unit 151 when an audiorecording function execution request, for example, an audio recordingfunction, is carried out according to the user's input. The audiorecording screen may include an indicator 1102 indicating a functionexecution state and a progress bar 1104 indicating a recording progressstate.

The indicator 1102 indicating a function execution state reflects acurrent function execution state such as “Recording” or “Playing”, forexample.

The progress bar 1104 may include an indicator 1106 indicating a starttime point of recording and an indicator 1108 indicating a currentrecording time point to indicate a recording progress state, and thedisplay unit 151 displays time information 1112 indicating a currentrecording time along with the progress bar 1104.

Referring to FIG. 3A along with FIG. 3B, for the next step, the userinput unit 130 (refer to FIG. 14 below) receives an input for selectinga predetermined portion of the item indicating the progressive statefrom the user (S1130). For example, the user input unit 130 receives aninput selecting a predetermined portion of the item indicating theprogressive state while receiving an audio signal.

To this end, the mobile terminal 100 can monitor whether a previous timepoint reproduction command is received. The previous time pointreproduction command may be a command for allowing the user to select aspecific time point corresponding to a time point prior to a currenttime point on the progress bar. Otherwise, it may be a command forselecting a time point prior to a specific time from a current timepoint, such as 10 seconds ago, 1 minute ago, or the like.

As an example of the selection command, the user may touch any one oftime points prior to a current recording time point on the progress bar1104, or drag the indicator 1106 indicating a recording start time pointor the indicator 1108 indicating a current recording time point to anyone of time points prior to a current recording time point. In thiscase, the display unit 151 may display another indicator 1110 indicatingthe selected time point (reproduction time point).

Referring to FIG. 3A again, for the next step, the controller 180reproduces an audio signal corresponding to the selected portion (orselected text string) (S1140). As the controller 180 reproduces theaudio signal, an audio signal corresponding to the selected portion maybe auditorily output through an earphone, a speaker or a receiver, or atext converted from the voice may be visually displayed through thedisplay. Furthermore, the display unit 151 may display the timeinformation of an audio signal corresponding to the selected portion.

For example, as illustrated in FIG. 3B(b), when a time point prior to acurrent recording time point is selected, the controller 180 reproducesthe recorded audio from the selected time point. Furthermore, thedisplay unit 151 changes the indicator 1102 indicating a functionexecution state to indicate that recording and playing are carried outat the same time, such as “Recording & Playing”, for example.Furthermore, the indicator 1110 indicating the selected time point(reproduction time point) moves on the progress bar 1104 to reflect areproduction time point on the display unit 151. Furthermore, thedisplay unit 151 may display time information 1114 indicating thereproduction time point along with the progress bar 1104.

For another example, when the user selects a time point prior to acurrent time point as illustrated in FIG. 3C(a), the display unit 151may display a text 1118 converted from a recorded voice from therelevant time point as illustrated in FIG. 30( b).

On the other hand, the controller 180 does not suspend currentlyprogressing audio recording while reproducing the recorded audio from aprevious time point. In other words, the controller 180 may reproducethe recorded audio at a previous time point while recording audio at acurrent time point, and the memory 160 may continuously store an audiosignal being input while reproducing an audio signal corresponding tothe selected portion. Accordingly, the mobile terminal 100 monitorswhether an audio recording end command is received, and terminates audiorecording upon receiving the audio recording end command.

For another example, referring to FIG. 3D, the mobile terminal 100 mayconvert a voice being recorded into a text based on STT to display theconverted text on the display unit 151. For example, as illustrated inFIG. 3D(a), an STT-based text for an audio signal that has been receivedfrom the start of recording up to a current time point may be displayedon the display unit 151.

However, the present disclosure is not limited to this. For example, thecontroller 180 may display an STT-based text for an audio signal thathas been received from a predetermined previous time point up to acurrent time point or display a representative text string for eachspecific unit (for example, paragraph) of an STT-based text for an audiosignal that has been received from the start of recording up to acurrent time point. Otherwise, the display unit 151 may display an itemindicating a progressive state in which the audio signal is being storedalong with an STT-based text for the stored audio signal.

In this case, as illustrated in FIG. 3D(b), the displayed text is formedto be scrolled, and thus the user can retrieve his or her desiredportion while recording is being progressed. Furthermore, the user inputunit 130 is formed to receive the selection of a partial text string orkeyword from the STT-based text displayed on the display unit 151. Theselected text string or keyword may be used for the setting of a filename, a bookmark or the like, and they will be described later.

According to a first operation control process of the foregoing firstembodiment, the user conveniently can check the content of previousrecording without terminating audio recording during the audiorecording. Furthermore, the user can control recording and reproductionat the same time through an intuitive user interface using a progressbar or text.

FIGS. 4A through 4D are views illustrating a second operation controlprocess of a mobile terminal according to a first embodiment disclosedherein.

In general, there is a portion desired to be particularly memorized orhighlighted by the user during the audio recording process. However, theprocess of retrieving the content of an audio file is not intuitive (inthe aspect of auditory characteristics compared to visualcharacteristics), and as a result, there may exist difficulty inretrieving the relevant portion again later. Furthermore, it may beburdensome to store the recording time of a portion desired to bememorized or highlighted in advance and retrieve it later.

Accordingly, the second operation control process is related to a methodof writing a specific time point through an input with a familiarpattern that can be instantaneously applied to a portion desired to bememorized or highlighted by the user during the audio recording.

FIG. 4A is a flow chart illustrating a process of writing a specifictime point during the audio recording, and FIG. 4B is a conceptual viewillustrating the process of FIG. 4A. FIG. 4C is a flow chartillustrating a process of reproducing audio data from the writtenspecific time point during the audio recording, and FIG. 4D is aconceptual view illustrating the process of FIG. 4C.

Referring to FIG. 4A, when the controller 180 starts audio recording,the controller 180 converts an audio signal being input to store it asaudio data (S1210). At this time, the display unit 151 may display aprogress bar indicating a recording state. The progress bar may includean indicator indicating a current time point of the whole recordedportion. When the whole recorded portion is not revealed, the indicatorindicating a current time point is displayed at the middle of theprogress bar.

Furthermore, the display unit 151 may display a speech-to-text(STT)-based text for an audio signal being input. For example, thedisplay unit 151 may display an STT-based text for an audio signal thathas been received from the start of recording up to a current timepoint, or display an STT-based text that has been received from apredetermined previous time point up to a current time point, or displaya representative text string for each specific unit (for example,paragraph) of an STT-based text for an audio signal.

Referring to FIG. 4B, the mobile terminal 100 executes an audiorecording function according to an user input and displays an audiorecording screen. The audio recording screen may include an indicator1202 indicating a function execution state and a progress bar 1204indicating a recording state.

The indicator 1202 indicating a function execution state reflects acurrent function execution state such as “Recording”, for example.

The progress bar 1204 may include an indicator 1206 indicating a starttime point of recording, and an indicator 1208 indicating a currentrecording time point to indicate a recording progress state, and theelectronic device, for instance, mobile terminal 100, displays timeinformation 1210 indicating a current recording time along with theprogress bar 1204.

Referring to FIG. 4A again, the input unit (for example, microphone 122,user input unit 130, sensing unit 140 (refer to FIG. 14 below), etc.)receives an input with a specific pattern while receiving an audiosignal (S1220). The input unit may receive the predetermined patterninput through any one of a touch sensor for sensing a specific region orbutton, a keypad, a tactile sensor, an acceleration sensor, a gyrosensor, a microphone, and the like.

The predetermined pattern input may include a touch input to a specificregion, an input to a specific key button (a push against a push buttonor a touch against a touch key), a change of tactile sense in a specificregion, a change of acceleration of the mobile terminal, a change ofangular speed of the mobile terminal, and a voice for which theintensity or volume of a specific voice command is greater than athreshold value, for example. Alternatively, the input unit may receivea selection input to part of an STT-based text displayed on the displayunit 151.

As an example of the predetermined pattern input, referring to FIG.4B(a), the mobile terminal 100 may sense a touch against a specificregion of the screen 121 of the mobile terminal 100 during the audiorecording.

Alternatively, the controller 180 may write a position of part of a textselected from the STT-based text displayed on the display unit 151 inthe recording information of audio data (refer to FIG. 1). The mobileterminal 100 may convert a voice being recorded into a text based on anSTT conversion to display the converted text on the display unit 151. Inthis case, a text string or keyword may be formed to be selected fromthe displayed text. For example, when a touch input against to anykeyword is applied, a position of the touched keyword may be written inthe recording information of audio data.

Referring to FIG. 4A again, the controller 180 reflects a time point atwhich the specific pattern is input to the recording information ofaudio data (S1230). The mobile terminal 100 may store a time point atwhich the relevant input is received along with audio recording.

Furthermore, the controller 180 may generate audio data and therecording information of the audio data as a file, or generate them asseparate files, respectively, to store them in the memory 160. Whenaudio data and the recording information of the audio data are generatedas a file, the recording information may be the meta data of an audiofile, and the audio data may be contained in the body of an audio file.When audio data and the recording information of the audio data aregenerated as separate files, a file in which the audio data is storedmay be reproduced with reference to a file in which the recordinginformation is stored.

Referring to FIG. 4B(b), the mobile terminal 100 may display anindicator 1214 indicating a time point at which a predetermined patternis input to the progress bar 1204. Otherwise, the mobile terminal 100may display an indicator indicating that a time point of receiving aninput with a predetermined pattern has been written instead ofdisplaying the indicator 1214 indicating a time point at which apredetermined pattern is input to the progress bar 1204. The mobileterminal 100 may output an indicator indicating that a time point ofreceiving an input with a predetermined pattern has been written in avisual, auditory, or tactile manner.

When writing for a specific time point is carried out during recordingas described above, an audio file stored by terminating recording ismade to notify information related to the specific time point to theuser. Hereinafter, the operation will be described in more detail.

Referring to FIG. 4C, when there is an access request for audio datastored through the processes S1210 through S1230 as illustrated in FIG.4A, the controller 180 acquires a time point at which a specific patternis received from the recording information of the requested audio data(S1240).

Furthermore, the controller 180 controls the display unit 151 to displayan indicator indicating the acquired time point at which a specificpattern is received (S1250). At this time, the display unit 151 displaysan indicator indicating a reproduction state of the audio data on theprogress bar.

Referring to FIG. 4D(a), when an audio data reproduction function iscarried out according to a request for executing the audio datareproduction function, for example, a user input, the mobile terminal100 executes the audio data reproduction function and displays an audiodata reproduction screen.

The audio data reproduction screen may include an indicator 1202indicating a function execution state and a progress bar 1204 indicatinga reproduction state. The indicator 1202 indicating a function executionstate reflects a current function execution state such as “Playing”, forexample.

The progress bar 1204 may include an indicator 1206 indicating a starttime point of recording, and indicators 1214 to 1208 indicating at leastone time point at which an input with a predetermined pattern isreceived, and an indicator 1208 indicating a current reproduction timepoint, the mobile terminal 100 displays time information 1210 indicatinga whole reproduction time along with the progress bar 1204.

Referring to FIG. 4C again, the user input unit 130 receives an inputfor selecting an indicator displayed on the display unit 151 from theuser (S1250). Then the controller 180 reproduces audio data from a timepoint corresponding to the indicator selected by the user (S1260).

Referring to FIG. 4D(b), the mobile terminal 100 receives an input forselecting any one indicator 1214 of the indicators 1214 to 1218indicating time points at which at least one predetermined pattern isreceived on the progress bar 1204. The mobile terminal 100 reproducesaudio data from a time point corresponding to the selected indicator. Inthis case, the indicator 1208 indicating a current reproduction timepoint may be an indicator indicating the selected time point.

As described above, according to a first embodiment disclosed herein, itallows the user to easily write a specific time point during the audiorecording. Accordingly, the facilitation of retrieval can be providedlater.

FIGS. 5A through 5D are views illustrating a third operation controlprocess of a mobile terminal according to a first embodiment disclosedherein.

In general, the mobile terminal 100 having a touch screen may provide ahandwriting function using a stylus or finger. Here, handwriting is abehavior for allowing the user to write the content of a lecture, adiscourse, a talk or the like, and the electronic device may store atouched position to provide a handwriting function. However, when theuser records audio while at the same time writing the content of alecture, a discourse, a talk or the like, the writing file and audiorecording file are separately managed, and thus there is a burden thateach file should be called when listening the content of the lecture,discourse, talk or the like, again later.

As a result, the third operation control process is related to a methodin which writing and audio recording are stored in synchronization witheach other and thus the content thereof can be conveniently checkedagain at the same time later.

FIG. 5A is a flow chart illustrating a process of synchronizing audiorecording with handwriting, and FIG. 5B is a conceptual viewillustrating the process of FIG. 5A.

Referring to FIG. 5A, the mobile terminal 100 first starts handwriting(S1310), and checks whether an audio recording command is received(S1320).

Upon receiving the audio recording command, the electronic device, forinstance, mobile terminal 100, performs audio recording insynchronization with handwriting (S1330) until a handwriting end commandis received (S1340). Upon receiving the handwriting end command, themobile terminal 100 terminates handwriting (S1350).

As a specific example, referring to FIG. 5B, the mobile terminal 100 maydisplay a handwriting screen when a handwriting function executionrequest, for example, a handwriting function is carried out according tothe user's input.

The handwriting screen may include a region 1302 in which handwriting isreceived and an audio recording menu 1304. The region 1302 in whichhandwriting is received displays a position touched using a stylus pen,a finger or the like. Accordingly, the content of handwriting (1306) canbe displayed on the region 1302 in which handwriting is received.

When the user selects audio recording menu 1304 while handwriting ismade, the mobile terminal 100 records a voice 1308 uttered by the userin synchronization with a handwriting progress time point.

The mobile terminal 100 can record a voice 1308 uttered by the user insynchronization with a handwriting progress time point in various ways.For example, the voice 1308 uttered by the user may be recorded tocorrespond to a time point at which handwriting is made in real time.Otherwise, the voice 1308 uttered by the user may be recorded tocorrespond to the unit of handwriting, for example, each page.

FIG. 5C is a flow chart illustrating a process of reproducing audiorecording in synchronization with handwriting during the reproduction ofhandwriting, and FIG. 5D is a conceptual view illustrating the processof FIG. 5C.

Referring to FIG. 5C, the mobile terminal 100 starts the reproduction ofhandwriting (S1370). Furthermore, the mobile terminal 100 checks whetheran audio recording reproduction command is received (S1380).

Upon receiving the audio recording reproduction command, the mobileterminal 100 reproduces audio recording in synchronization withhandwriting being reproduced (S1390). However, when the audio recordingreproduction command is not received, the mobile terminal 100 reproducesonly handwriting without reproducing audio recording (S1395).

Referring to FIG. 5D, when a handwriting reproduction function executionrequest, for example, a handwriting reproduction function is carried outaccording to the user's input, the mobile terminal 100 may execute ahandwriting reproduction function and display a handwriting reproductionscreen. The handwriting reproduction screen may include a region 1310 inwhich handwriting is reproduced and a reproduction menu 1312.

When the reproduction menu 1312 is selected, the mobile terminal 100displays the content of handwriting 1314 written by the user using astylus pen, a finger, or the like in the region 1310 in whichhandwriting is reproduced in the handwriting unit, for example, pageunit. In this case, the mobile terminal 100 reproduces audio recording1316 stored in synchronization with the content of handwriting 1314,namely, a voice uttered by the user while writing the content ofhandwriting.

As describe above, according to a first embodiment disclosed herein,handwriting and audio recording are store in synchronization with eachother, thereby allowing the user to conveniently check the content atonce. As a result, it may be possible to enhance the effectiveness ofeducation.

A mobile terminal according to the present disclosure is implemented toprovide a file name using the converted text. Hereinafter, the processassociated with providing a file name will be described in detail as asecond embodiment.

FIG. 6A is a flow chart illustrating a method of automatically providingthe file name of an audio file recorded according to a second embodimentof the present disclosure based on an STT-based text.

Referring to FIG. 6A, according to a second embodiment of the presentdisclosure, the mobile terminal performs a speech-to-text (STT) functionor algorithm for a voice being recorded or recorded voice to acquire atext, a text string or a script. Furthermore, the mobile terminal maydisplay the text, text string, or script on the display unit uponreceiving an access request for a list of audio files from the user. Itwill be described below in detail.

First, audio recording is started (S2110). When the audio recording isstarted, audio data being received in real time is stored in a buffer.The buffer may be a medium with the highest processing speed, forinstance, a volatile memory. Otherwise, the buffer may be an internalmemory of the electronic device, for instance, mobile terminal.

At this time, audio data stored in the buffer may be stored as atemporary audio file with a constant period. The temporary audio filemay be stored in a medium with the highest processing speed, forinstance, a volatile memory. A temporary file name may be provided tothe temporary audio file according to a randomly provided number orpredetermined rule.

Next, real-time speech-to-text (STT) is carried out to acquire a text, atext string or a script (S2120). The STT recognizes a word (forinstance, a subject, a predicate, a noun, a verb) from a voice asdescribed above, and arranges the recognized words to generate a text, atext string or a script.

When the audio recording is terminated (S2130), an audio file isgenerated (S2140). The audio file is generated based on the temporaryaudio file. For example, the audio file may be stored in a medium with alarge storage capacity, the medium with a large storage capacity may below speed. Accordingly, the audio file may be generated in the mediumwith a large storage capacity based on being moved from the temporaryaudio file stored in the medium with a high processing speed.

In this case, meta data may be generated along with the audio file. Themeta data may include a recording date, a total recording time, alocation obtained from GPS, a name of speaker, a contact address, andthe like. The location obtained from GPS is obtained as a coordinatevalue, and thus the mobile terminal may obtain a geographic name througha map data service based on the coordinate value, and then store thegeographic name as meta data.

The name of speaker may be obtained a name called by the other betweenspeakers. For example, assuming a circumstance in the meeting, whenspeaker A converses while calling the other Alice, and speaker Bresponds to the conversation, the mobile terminal 100 derives that thename of the speaker B is Alice through the conversation. Furthermore,speaker B converses while calling the other Bob in the state ofremembering the voice of the speaker A and the voice of the speaker B(for example, in the state of remembering his or her voice tone, mannerof speaking, and the like), and the speaker A responds to theconversation, the mobile terminal 100 derives that the name of thespeaker A is Bob through the conversation.

When the names of speakers are acquired through the conversation asdescribed above, the electronic device, for instance, mobile terminal100, may store the names of the speakers as meta data, and acquire thecontact addresses of the speakers from an address book to store thecontact addresses along with the meta data.

On the other hand, when the audio file is generated, the mobile terminal100 may perform STT based on the generated audio file to generate a text(S2150). When the text is generated in the foregoing step S2120, thestep S2150 may be omitted. On the contrary, only the step S2150 may becarried out without the step S2120.

Otherwise, the step S2120 is carried out in real time, and the voice maynot be recognized in real time according to circumstances, and thus thestep S2150 may be carried out to process such an incomplete portion.

For example, when STT is carried out in the step S2120 in a circumstancethat a plurality of speakers converse with one another, it may bedifficult to recognize the voices of the plurality of speaker in realtime. Accordingly, in such a circumstance, only the voices of somespeakers may be recognized to acquire a text in the step S2120, and thevoices of the other speakers may be recognized to acquire a text in thestep S2150. Subsequently, the text obtained from the voices of somespeakers and the text obtained from the voices of the other speakers arecombined to generate a text. At this time, when combining the two texts,a text string based on the voice of each speaker may be arranged in theorder of time.

For another example, when the mobile terminal 100 performs STT in realtime to obtain a text in a circumstance that a plurality of speakersconverse at the same time, the conversation of two persons may berecognized in a mixed manner. As a more specific example, when speaker Autters “What about having a meeting again next week?”, and speaker Butters “Let's make a schedule next week” at the same time, the mobileterminal 100 may recognize the conversation as “What about let's makehaving a meeting again a schedule next week”, thereby generating anerror. As a result, the mobile terminal 100 may recognize speaker A andthen perform STT only for the voice of the speaker A to obtain a text inthe step S2120, and perform STT only for the voice of the speaker B toobtain a text in the step S2150, and then combines the texts of thespeakers A and B.

Subsequently, the mobile terminal 100 may provide a file name of theaudio file based on the text (S2160).

At this time, in the related art, the file name is typically generatedbased on a general rule. For example, files having file names such asAUDIO_001, AUDIO_002, AUDIO_003, and the like are generated by appendingnumerals in the order. Otherwise, in the related art, the file name ofeach recorded file is generated with a combination of the recording datewith a serial number. In other words, the foregoing three files may begenerated as 20120112_001, 20120212_001, 20120312_001, and the like,according to the related art.

However, when the mobile terminal displays such a file name according toan access request for a list of audio files, the user does not know atall what type of recording has been made with only the relevant filename.

According to the present disclosure, when recording is terminated, afile name for the audio file is provided based on a text converted andgenerated based on a voice within the audio file. For example, thecontroller detects a core keyword from the text, and sets the detectedcore keyword as at least part of a file name for the audio file.

The file name of the audio file may be automatically provided based onthe converted text or provided according to the user's selection.Hereinafter, a case of automatically providing a file name will bedescribed with reference to FIGS. 6B through 6D, and a case of partiallyadding the user's selection will be described with reference to FIGS. 6Eand 6F, and a case of providing a file name according to the user'sselection will be described with reference to FIG. 6G.

FIG. 6B illustrates an example in which a file name is automaticallyprovided to an audio file.

Referring to FIG. 6B, three audio files are shown, and they areprocessed to detect core keywords according to different methods,respectively.

A first audio file indicates a case in which the core keyword is a worduttered by a specific speaker at a specific time point (for example,during the introduction of a conversation). For example, in a generalmeeting, the host typically speaks a matter when opening the meeting.Accordingly, when speaker A utters “Let's start a new productdevelopment meeting for mobile phones” during the introduction of aconversation, the mobile terminal 100 may detect “meeting” as a corekeyword, and provides a text string containing the same “new productdevelopment meeting for mobile phones” as a file name. In this case, thespecific time point may be a range for a predetermined period of time(for example, one minute after starting recording), and thepredetermined period of time may be set by the user.

Otherwise, the core keyword may be a word emphasized by a specificspeaker during the recording. The emphasis may be defined through theemotion analysis of a voice or a repetition rate of the word.

A second audio file indicates a case in which a file name is providedthrough the emotion analysis of a voice. For example, the terminaldetects a keyword at a portion where a specific speaker speaks in anexcited state or speaks in a high tone voice using an emotion analysisalgorithm. Specifically, when the user records a LTE related articleamong news contents broadcast from a radio or television, the mobileterminal 100 analyzes emotions from the voice of a speaker, namely, anews anchor, to know whether it is a news. Furthermore, when the newsanchor speaks while emphasizing a word LTE, the mobile terminal 100 maydetect the word LTE as a core keyword, and extract “LTE related articlesscrap” as a text string to be set to the file name.

For another example, a word with a high repetition rate may be detectedas a core keyword, and in this case, a word with a high repetition rateat a specific section (for example, the introduction of a conversation,the end of a conversation) may be a core keyword. For example, when aphrase “antenna design” is reiterated several times while speakers A andB converse with each other, the antenna design may be provided as a filename. Moreover, when a word “meeting” is detected during theintroduction of a conversation, “antenna design meeting” may be providedas a file name (refer to the third file). In this manner, the foregoingexamples may be combined with one another.

Furthermore, the keyword may by extracted by recognizing a recordedplace or circumstance. For example, a keyword may be detected from theintroduction of a conversation when the mobile terminal is located at anoffice or customer's site and thus frequent meetings may be expected,and a word extracted through the emotion analysis may be detected as akeyword when the mobile terminal is located at home and thus frequenttelevision viewing are expected.

Hereinafter, an example of automatically providing the file name will bedescribed in detail with reference to FIGS. 6C through 6F.

FIG. 6C illustrate an example of automatically providing the file nameof an audio file according to the method illustrated in FIG. 6A with atable.

As illustrated in FIG. 6C, when audio files are generated through theforegoing step S2140, temporary file names AUDIO_001, AUDIO_002,AUDIO_003, and the like are provided based on serial numbers,respectively. Otherwise, when audio files are generated through theforegoing step S2240, temporary file names, for instance, 20110403_001,20110505_001, 201200404_001, and the like, may be provided based on acombination of the recording date with a serial number. In other words,the foregoing three files may be generated as according to the relatedart.

The examples of providing file names for three audio files in such acircumstance will be described below in detail.

According to the table, a temporary file name AUDIO_001 is provided forthe illustrated first audio file, and a text string made of corekeywords, namely, “new product development meeting for mobile phones”,is extracted from the STT-based text, and the date information is2011-04-03, and the attendees are Alice, Bob and Charlie, and thelocation as meta data is Yeouido in Seoul, and the contact addresses ofeach attendee are stored. Then, the mobile terminal 100 provides “newproduct development meeting for mobile phones” as a file name. Moreover,the mobile terminal 100 may automatically provide the file name as acombination of text strings made of the date and core keywords for thefirst audio file. For example, as illustrated herein, “20110403 newproduct development meeting for mobile phones” may be provided as a filename.

For another example, a temporary file name AUDIO_002 is provided for theillustrated second audio file, and a text string made of core keywords,namely, “LTE related articles”, is extracted from the STT-based text,and the date information is 2011-05-05, and the attendees are Bob andCharlie, and the location as meta data is Gangnam in Seoul, and thecontact addresses of each attendee are stored. Then, the mobile terminal100 provides “LTE related articles scrap” as a file name. Moreover, asillustrated herein, the mobile terminal 100 may combine text stringsmade of the date and core keywords for the second audio file to provide“20110505 LTE related articles scrap”, for example, as a file name.

For still another example, a temporary file name AUDIO_003 is providedfor the illustrated third audio file, and a text string made of corekeywords, namely, “antenna design meeting”, is extracted from theSTT-based text, and the date information is 2012-04-04, and theattendees are Charlie, Alice and Bob, and the location as meta data isMyeongdong in Seoul, and the contact addresses of each attendee arestored. Then, the mobile terminal 100 may provide “20120404 antennadesign meeting” as a file name using a combination of text strings madeof the date and core keywords as illustrated herein.

An example capable of distinguishing each audio file only with acombination of the date and core keywords has been illustrated in FIG.6C, but if there is a file with the same date and the same corekeywords, then the mobile terminal 100 may further use at least one ofthe attendees, recording start time, and location information to providea file name.

FIG. 6D illustrates another example for automatically providing the filename of an audio file according to the method illustrated in FIG. 6A.

Referring to FIG. 6D, the electronic device, for instance, mobileterminal 100 may use this year, last year, year before last, andyear/month/day when providing a file name without using the form ofYYYYMMDD.

Furthermore, the electronic device, for instance, mobile terminal 100,may use a geographic name when providing a file name.

For example, for the illustrated first audio file, the mobile terminal100 may provide a file name “new product development meeting for mobilephones in Yeouido last year”, for example, for the first audio file,using a combination of text strings made of the date, geographiclocation, and core keywords.

Here, it is noteworthy that the concept of last year/this year may bechanged as time passes by, and thus the mobile terminal 100automatically the file name according to the time elapse.

For example, if the present is year 2011, the second file is generatedin 2011-05-05, then the electronic device, for instance, mobile terminal100, may provide a file name “LTE related articles dated 05-05 inGangnam this year” for the second file.

However, if the present is year 2012 as time passes by, the electronicdevice, for instance, mobile terminal 100 may automatically update thefile name of the second file as “LTE related articles in Gangnam lastyear”. Otherwise, the automatic update may be carried out wheneverperforming the audio recording related function. Furthermore, theautomatic update may be scheduled to be carried out during an idle time.Otherwise, the automatic update may be scheduled to be carried out oncewhenever month or year is changed.

On the other hand, the month/day is not shown in the file name accordingto an example of the second file. It is because the mobile terminal 100determines that each audio file can be sufficiently distinguished onlyby file names made of the geographic name and core keywords. However,when it is determined that such distinction is impossible, the mobileterminal 100 may add even information on the month/day to the file nameas illustrated in the third file.

FIG. 6E illustrates a process of allowing the user to confirm afterautomatically providing the file name of an audio file according to themethod illustrated in FIG. 6A.

Referring to FIG. 6E, immediately after generating a third audio file,the mobile terminal 100 automatically provides a file name “20120404antenna design meeting” for the third audio file as described above.

In this manner, the automatically provided file name is based on a textstring made of core keywords extracted from the STT-based text by themobile terminal 100. Such core keywords may not be suitable according tocircumstances.

Accordingly, the mobile terminal 100 may automatically provide a filename, and then perform an operation for allowing the user to confirmwhether the provided file name is suitable. For example, as illustratedin FIG. 6E(a) or 6E(c), the mobile terminal 100 may provide the filename, and then display a “Modify” button and a “Confirm” button.

In this case, when the user touches the confirm button as illustrated inFIG. 6E(a), a file name “20120404 antenna design meeting” may beconclusively provided for the audio file as illustrated in FIG. 6E(b).ON the contrary, when the user touches the modify button as illustratedin FIG. 6E(c), the file name of the audio file is switched to amodifiable state. For such an example, as illustrated in FIG. 6E(d),another file name may be proposed. For the proposal of another filename, priorities may be provided for the created file names. Forexample, a repetition rate in the core keyword detection method isprovided as the priority, and an emotion analysis or specific time pointanalysis may be used as the next priority. In this case, as the userselects the modify button, the proposed another file name may be a textstring containing core keywords extracted by the next priority analysis.

Alternatively, subsequent to providing the file name, the mobileterminal 100 may display the file name with a slant effect or with a dimcolor. The mobile terminal 100 may release the slant effect or dim colorwhen the user touches the file name. However, when the file name istouched for more than a predetermined period time (for example, short orlong touch), the mobile terminal 100 may display a screen capable ofmodifying the file name.

Alternatively, the subsequent to providing the file name, the mobileterminal 100 may display a “Confirm” button while at the same timedisplaying the file name with a slant effect or with a dim color. Whenthe user touches the text string without touch the confirm button, themobile terminal 100 may display a screen capable of modifying the filename.

FIG. 6F illustrates an example in which the file name of an audio fileis automatically provided as illustrated in FIG. 6A but there exist aduplicate file name.

When the file name of an audio file is automatically provided but thereexist a duplicate file name, the mobile terminal 100 may further displayother recommended file names. Otherwise, when there is a duplicate filename, the mobile terminal 100 may display other core keywords. When theuser selects any one of the other core keywords, the mobile terminal 100may update the file name using the selected keyword.

More specifically, when there previously existed a file name “antennadesign meeting”, the recorded and added file name may be provided with“antenna design meeting”. Referring to FIG. 6F, when there is aduplicate file name, the mobile terminal 100 may display an instructionphrase “Recommended file name is duplicate. Do you want to modify?”.Then the mobile terminal 100 may display a “Confirm” button or a “No”button.

Otherwise, when there is a duplicate file name, the mobile terminal 100may display a list of file names that can be provided. The list of filenames may be core keywords with the next priority, and the user maytouch his or her desired file name among the file names on the list toprovide a file name to the audio file.

FIG. 6G illustrates a case where the user creates an audio file name.

Referring to FIG. 6G, a text converted by STT is displayed on thedisplay unit 151 while recording is carried out. When recording iscarried out, a converted text may be displayed in real time, or only acore keyword or text string extracted from the text may be displayed.

The user may select a keyword to be provided as a file name among textsdisplayed on the display unit 151 while recording is carried out. Forexample, when the user drags to select a text string “antenna design”,the text string “antenna design” will be created as a file name later.

Such a selection input may be applied to a plurality of keywords or textstrings. For example, when the user drags “antenna design” and touches“meeting”, they are combined to create “antenna design meeting” as afile name.

Furthermore, a keyword or text string selected by the user's touch ordrag while displaying the text may be stored therein as a list forproviding a file name later. The list may be displayed as selectableitems when recording is terminated and stored.

The embodiment of FIG. 6G may be implemented by various types of drivingmethods. For example, the controller may implement the driving method bythe execution of three steps. More specifically, the controller drives arecording application for recording a user's voice input through themicrophone into an audio file during the first step, and drives aspeech-to-text (STT) execution application in linkage with driving therecording application to display a text converted and generated from theuser's voice on the display unit during the second step. The controllersets a keyword specified by a touch input from the content of the textdisplayed on the display unit to at least part of a file name for theaudio file during the final third step.

The controller may detect a core keyword from the text, and set thedetected core keyword to at least part of the file name when there is notouch input during the recording of the audio file. In this case, thefile name may be made of a text string containing the keyword or thecore keyword, and the controller may display the text string whilescrolling it when there is an output request for a list of the audiofiles.

Furthermore, the configuration of the application may be carried out invarious ways. For example, the STT execution application may beseparately provided from the recording application, and the recordingapplication may be carried out to allow the selection of the driving ofthe STT execution application during the recording. For another example,the STT execution application may be provided as part of the recordingapplication such that the execution of the STT is implemented as afunction of the recording application during the driving of therecording application.

Furthermore, when storing the audio file, a text file containing thetext generated by STT may be also stored along with the audio file.Through this, a text corresponding to the voice being reproduced can beimmediately displayed using the text file without a separate STT processduring the reproduction of the audio file later. For example, a popupwindow indicating the existence of the terminal folder may be displayedduring the reproduction of an audio file, and the text may be displayedin linkage with the reproduction of the audio file by the user'sselection.

FIG. 7A is a flow chart illustrating a method of modifying a file namethat has been automatically provided according to the modified exampleof the second embodiment of the present disclosure, and FIGS. 7B through7E illustrate an example of modifying a file name.

The processes S2210-S2260 illustrated in FIG. 7A are similar to theprocesses S2110-S2160 illustrated in FIG. 6A, and thus the foregoingdescription of FIG. 6A will be used as it is with no duplicatedescription.

When a file name modification request is received subsequent toautomatically providing a file name for the generated audio file(S2270), the mobile terminal 100 enters a modification mode to modifythe file name.

As illustrated in FIG. 7B, when the user takes a gesture for thespecific file name (for example, when the user performs a short or longtouch to a specific file name, or drags it in a specific direction), themobile terminal 100 may create and display an effect on the specificfile name. For example, the mobile terminal 100 may display the specificfile name with a slant effect, a dim effect, or a shaking effect.

The mobile terminal 100 may display a screen for modifying the file namewhile displaying the specific file name with an effect in this manner. Avirtual keyboard may be displayed for the screen for modification. Whenthe user input a specific word using the virtual keyboard as illustratedin FIG. 7B, the mobile terminal 100 may display the input keywords as afile name instead of the word.

On the other hand, referring to FIG. 7C, when the user takes a gesturefor the specific file name (for instance, the user performs a long touchto a specific file name or drags it in a specific direction), the mobileterminal 100 may display a “Modify” button and a “Confirm” button to askthe user whether he or she wants to modify it. When the modify button isselected by the user, the mobile terminal 100 may display a screen formodifying the file name.

As illustrated in the drawing, another recommended keyword with the nextpriority may be displayed on the screen for modification, and when theuser selects any one of the displayed other recommended keywords, themobile terminal 100 may updates the file name using the selectedkeyword.

The modification of the file name may be carried out by voicerecognition as illustrated in FIG. 7D. In this case, a screen forreceiving the voice recognition may be created with a phrase for guidinga voice input along with a microphone image as illustrated in FIG.7D(b). Moreover, though not shown in the drawing, a recognized voice maybe displayed on the screen for receiving the voice recognition.

Referring to FIG. 7D(a), when a list of audio files is accessed forediting through voice recognition, an icon corresponding to audioedition may be displayed at a position of the display unit 151. The iconmay be a microphone image disposed at a side of the file name.

When the user touches the icon, the screen is displayed, and when avoice is received from the user, the file name may be modified accordingto the received voice command.

For example, when the received voice command is “Would you change thefile name to mobile phone antenna design?”, the mobile terminal 100 maychange the existing file name to “mobile phone antenna design” accordingto the recognized voice.

For another example, when the received voice command is “Would youchange design to simulation?”, the mobile terminal 100 may replace onlysome words in the existing file name to a new word.

On the other hand, as illustrated in FIG. 7E, the screen for modifyingthe file name may be a window for displaying a text converted based onSTT.

For example, when a list of audio files is accessed, an iconcorresponding to the output of the text may be displayed as illustratedin FIG. 7E. When the user touches the icon, a text will be displayed onthe display unit 151, and the displayed text may be generated to bescrolled by a drag or the like. At this time, as illustrated in thedrawing, when the user selects a specific keyword or text string amongthe content of the text through a touch or drag, a confirmation screenshowing whether to change the file name is displayed, and the file nameis changed when the user accepts the file name change.

An embodiment disclosed in FIG. 7E may be applicable to setting a firsttime file name as well as changing a file name. For example, a textconverted by STT may be displayed on the screen while recording voice,and when the user selects a specific keyword or text string from thecontent of the text through a touch or drag, the selected keyword ortext string may be provided as a file name.

As described above, according to a second embodiment of the presentdisclosure, it may be possible to automatically provide the file name aswell as conveniently change the file name through a keyword extractedfrom the STT-based text.

Furthermore, when accessing a file list, the mobile terminal 100 maydisplay additional information in addition to the file name. With regardto this, a third embodiment disclosed herein will be described withreference to FIGS. 8A through 8J.

FIG. 8A is a flow chart illustrating a method of displaying a textassociated with the audio file according to a third embodiment of thepresent disclosure.

The processes S2310-S2350 illustrated in FIG. 8A are similar to theprocesses S2110-S2150 illustrated in FIG. 6A, and thus the foregoingdescription will be used as it is with no duplicate description, andtherefore, they will be described from the process S2360.

When an access request for a list of the audio files (or audio fileitself) is received subsequent to generating an audio file (S2360), atext string may be described based on a text corresponding to each audiofile on the display unit (S2370).

The access request may be the user's request for a list of the audiofiles. Otherwise, the mobile terminal 100 may automatically generate theaccess request in response to a termination request of the audio file.

Alternatively, when a request for executing an audio recordingapplication again is received in a state that the audio recordingapplication is terminated, the mobile terminal 100 may first display alist of the audio files. Accordingly, the access request may be arequest for executing the audio recording application.

According to a third embodiment of the present disclosure, the mobileterminal may extract a suitable text string from the text generatedbased on STT, and display the extracted text string in response to anaccess request for the audio file. At this time, a method similar to orsame as the foregoing second embodiment will be used for the text stringextraction method.

When it is difficult to display all the extracted text string on onescreen, the mobile terminal 100 may display the text string while beingautomatically scrolled. In other words, the mobile terminal 100 may showthe text string on the right side and then move it in the leftdirection, thereby displaying the entire text string. In other words,the text string is displayed as many as the number of characters thatcan be displayed as a line on the screen, and then moved in a horizontaldirection to display the remaining characters of the text string.Otherwise, the first character of the text string is shown at one sideof the screen, and then moved to the other side thereof to continuouslydisplay the remaining characters of the text string.

Examples of the foregoing process will be described in more detail withreference to FIGS. 8B through 8J.

FIG. 8B illustrates an example of displaying a text string fordescribing an audio file.

Referring to FIG. 8B, three audio files are displayed thereon. A firstaudio file is generated at 15:30, Jan. 12, 2012, and recorded for total20 minutes 10 seconds, and the file name is “new product developmentmeeting for mobile phones”.

A second file name is “LTE related articles scrap”, and a third filename is “antenna design meeting”.

As illustrated in the drawing, a different text string from the filename may be displayed along with the file name of an audio file. Thecontent containing the file name or the content containing a corekeyword that is not contained in the file name may be written in thetext string.

A different text string from the text string set to the file name may becreated by varying the detection condition. For the detaileddescription, it is illustrated a case in which a file name “antennadesign meeting” is created using the same method as the file name of athird audio file according to a second embodiment.

When phrases “next month overseas trip” and “component suppliercontract” are reiterated several times in addition to “antenna designmeeting” while speaker A converses with speaker B, the mobile terminal100 may recognize it as a preliminary meeting prior to overseas tripthrough the phrase “next month overseas trip”, and furthermore,recognize that the meeting is to prepare a contract draft through aphrase “contract draft preparation”. Furthermore, the mobile terminal100 may generate “preliminary meeting prior to overseas trip: contractdraft preparation” as a text string to be displayed along with the filename.

On the other hand, when the electronic device, for instance, mobileterminal 100, displays a portrait screen, in case where the entire textstring (“preliminary meeting prior to overseas trip: contract draftpreparation”) cannot be displayed on the portrait screen, the textstring may be shown in the right side and then moved in the leftdirection, thereby displaying the entire text string.

For example, as in the illustrated third audio file, “preliminarymeeting prior to overseas trip” may be shown first, and then the textstring may be scrolled, i.e., moved in the left direction, and thus“contract draft preparation” may be subsequently displayed.

FIG. 8C illustrates another example of displaying a text string fordescribing the audio file.

Referring to FIG. 8C, when the user takes a gesture for a specific audiofile when the mobile terminal 100 scrolls the text string in acircumstance that the entire text string for an audio file cannot bedisplayed on one screen, the scrolling speed of the text string can becontrolled.

Specifically, it is illustrated that the mobile terminal 100 firstdisplays “LTE related articles SBS radio” for the illustrated secondaudio file and then moves it in the left direction and finally displaysa subsequent portion of “radio”.

At this time, when the user touches or drags the corresponding portionof the screen as a gesture for the second audio file, the mobileterminal 100 may display the text string by decreasing or increasing thescrolling speed, namely moving speed. Specifically, when the usertouches the second audio file, the mobile terminal 100 may decrease thescrolling speed, thereby allowing the user to more accurately read thetext string. Otherwise, when the user drags the second audio file, themobile terminal 100 may increase the scrolling speed. For example, thescrolling speed may be increased in case of dragging as illustrated inFIG. 8C(b) than a case of touching as illustrated in FIG. 8C(a).

As another specific example, when the user drags the second audio filein the right direction, the mobile terminal 100 may move the text stringin the opposite scrolling direction.

On the other hand, when the user indicates the corresponding portion ofthe screen with his or her finger without touching or dragging thecorresponding portion of the screen as a gesture for the second audiofile, the mobile terminal 100 may recognize the user's finger gesturethrough the camera, and decrease the scrolling speed according to thefinger gesture. Furthermore, when the user indicates the correspondingportion of the screen with his or her finger and then moves it in theleft or the right direction as a gesture for the second audio file, themobile terminal 100 may increase the scrolling speed or move the textstring in the opposite scrolling direction.

FIG. 8D illustrates another example of displaying a text string fordescribing the audio file.

Referring to FIG. 8D, the user may access the text or text string of anaudio file with his or her voice in a state that three audio files aredisplayed on the screen of the mobile terminal. It will be describedbelow in detail.

As illustrated in the drawing, text strings “new product developmentmeeting for mobile phones”, “LTE related articles scrap”, “antennadesign meeting” are displayed on the screen of the mobile terminal, anda text string being scrolled “article SBS radio” is displayed in the“LTE related articles scrap”. Furthermore, in this case, the file nameitself may be scrolled as in the third file. For another example, thoughnot shown in the drawing, the file name may be created such as AUDIO_001by appending a numeral other than a core keyword, and a text stringdisplayed along with the file name may be displayed while beingscrolled.

At this time, when the user utters “Please summarize and speak thesecond voice memo”, the mobile terminal 100 may recognize theconversation command, and display core keywords extracted from thecorresponding text of the second audio file on the screen or output themwith a voice. The output voice may be one for which the extracted corekeywords are converted into a voice through a text-to-speech (TTS)conversion. Otherwise, the output voice may be one for which onlyportions related to the keywords are extracted from the audio file andthe extracted portions are combined with one another.

In this case, the mobile terminal 100 may display core keywords on thescreen or output them with a voice through the following process. Forexample, when the conversation command is recognized, the mobileterminal 100 may perform STT and then display the recognized textstring, i.e, “Please summarize and speak a second voice memo” on aportion of the screen, for instance, on a lower end of the screen.Subsequently, the electronic device, for instance, mobile terminal 100may output a question “Is it right for your request?” through thespeaker. Otherwise, the mobile terminal 100 may output a question “Is itright that you have requested to summarize and speak a second voicememo?

Otherwise, alternatively, when the user utters “Please speak about LTErelated articles in detail”, the electronic device, for instance, mobileterminal 100, may recognize the conversation command and perform STT tooutput “Do you want to have the detailed content of LTE relatedarticles?” with a voice or display it on the screen.

When the voice of an affirmative expression (or position expression, forinstance, Okay, Yes, Thank you, etc.) is received subsequent tooutputting the above question, the mobile terminal 100 may display corekeywords extracted from a text corresponding to the second audio file onthe screen or output them with a voice.

Otherwise, even when there is no input from the user for a predeterminedperiod of time subsequent to outputting the question, the mobileterminal 100 may analyze it as an affirmative expression.

FIGS. 8E and 8F illustrate another example of displaying a text stringfor describing the audio file in detail.

Referring to FIG. 8E, when the user takes a gesture for a specific textstring in a state that text strings for three audio files are displayedon the screen of the mobile terminal, the electronic device, forinstance, mobile terminal 100 may display the column to which therelevant text string belongs with a larger width, and display a moredetailed description for the text string in the relevant column. It willbe described below in detail.

As illustrated in the drawing, text strings “new product developmentmeeting for mobile phones”, “LTE related articles scrap”, “antennadesign meeting” are displayed on the screen of the mobile terminal, anda text string being scrolled “article SBS radio” is displayed in the“LTE related articles scrap”.

At this time, when the user touches the column to which the second textstring belongs for more than a predetermined period of time on thescreen, the mobile terminal 100 may display the column to which thesecond text string belongs with a larger width. At this time, a width ofthe column to which the second text string belongs is widened and as aresult, the third column may be moved toward the lower portion thereofor may not displayed on the screen.

Subsequently, the mobile terminal 100 may summarize and display anSTT-based text corresponding to the second audio file within the secondcolumn with a larger width. At this time, when all the summarizedcontent cannot be displayed within the second column with a largerwidth, the mobile terminal 100 may display a scroll bar on the rightside.

On the other hand, such a method may be also enabled by another gestureother than performing a long touch to the column to which the secondtext string belongs on the screen by the user.

For example, even when the user touches the column to which the secondtext string belongs and then drags it in the bottom direction or dragsit in the top direction on the screen, the mobile terminal 100 maydisplay the column to which the relevant text string belongs with alarger width, and display a more detailed description for the textstring in the relevant column.

Referring to FIG. 8F, when the user touches the column to which therelevant text string belongs with his or her two fingers, and then dragsit in a direction that the touched two fingers are moved away from eachother, the mobile terminal 100 may display the column to which therelevant text string belongs with a larger width, and display a moredetailed description for the text string in the relevant column.

In this case, a rate of the width of the column being widened may becontrolled according to the dragging speed. For example, when thedragging speed is high as illustrated in FIG. 8F(b), the column may beenlarged more than a case that the dragging speed is low as illustratedin FIG. 8F(b). Moreover, when the dragging speed is greater than apredetermined size, the other columns may be disappeared to display therelevant text string on the entire screen.

FIG. 8G illustrates another example of displaying a text string fordescribing the audio file in detail.

As illustrated in FIG. 8G, when the detailed description of a textstring for the second audio file is displayed, the electronic device,for instance, mobile terminal 100 may display a core keyword of thedetailed description with a bold or underline attribute.

Otherwise, the mobile terminal may process and display the core keywordwith a highlight or red attribute.

In general, when displaying the detailed description, if the content isarranged based on the core keyword, then the user may be unable toreconstruct and memorize the circumstance of the recording time.Accordingly, as illustrated in FIG. 8F, when displaying the detaileddescription, the mobile terminal 100 may display all the sentencecontaining the core keyword as well as the core keyword, and display therelevant core keyword with a bold or underline attribute, therebyallowing the user to reconstruct and memorize the circumstance of therecording time. At this time, when the user performs a touch to the corekeyword with a bold or underline attribute, only the relevant portionmay be reproduced from the audio file to output a voice. Otherwise, theportion may be created as a file name to be changed.

Alternatively, when displaying the detailed description, the mobileterminal 100 may arrange and display only the keywords. At this time,the keyword may be displayed with an underline attribute to indicate alink. In this case, when the user touches the relevant word on thescreen, the mobile terminal 100 may reproduce only the relevant portionfrom the audio file while displaying all the sentence containing thecore keyword.

For another example, when displaying the detailed description, if thereare terms that are not typically used such as technical terms, then themobile terminal 100 may process and display the relevant term with anenhancement effect, for instance, a bold, underline, highlight, or redattribute.

At this time, when the user touches the enhanced word on the screen asdescribed above, the electronic device, for instance, mobile terminal100 may display a prior search result of the word on the pop-up screen.

FIG. 8H illustrates another example of displaying an STT-based textcorresponding to the audio file in detail on the entire screen.

When there is a gesture, for instance, double touch, to the foregoingaudio file from the user, the mobile terminal 100 may display the wholeSTT-based text corresponding to the second audio file as illustrated inthe drawing.

At this time, when the whole text cannot be all displayed on the screen,the electronic device, for instance, mobile terminal 100 may display ascroll bar on the right side thereof. When the whole text cannot be alldisplayed on the screen, the electronic device, the mobile terminal 100may display a scroll bar on the right side thereof while displaying thewhole text from the beginning, but alternatively, the mobile terminal100 may display it from a portion containing a core keyword in the wholetext on the screen. In this manner, it enables the user not tounnecessarily read a portion such as greetings typically made byspeakers when recording is carried out.

As described above, the core keyword may be displayed with anenhancement processing, for instance, bold, underline, highlight, or redattributes, and when the user touches the enhanced core keyword, therelevant portion of an audio file may be output with a voice.

FIG. 8I illustrates an example of allowing the user to send a commandwith a voice in a state that an STT-based text corresponding to theaudio file is displayed in detail on the entire screen.

As described above, the mobile terminal 100 may display a screen forreceiving a voice command from the user at a lower end portion thereofwhile displaying the whole STT-based text.

At this time, when the user utters “Would you play from the LTE standardpatent portion?”, the mobile terminal 100 recognizes the voice commandand reproduce a voice from the relevant portion.

In this case, the mobile terminal 100 may perform STT prior toreproducing the voice to output the recognized text string “Would youplay from the LTE standard patent portion?” with a voice or display iton the screen.

On the other hand, the display of the whole text may be controlled ininterlock with the reproduction of a voice. For example, a textconverted based on STT may be displayed on the display unit 151 alongwith the reproduction command of a voice as illustrated in FIG. 8J. Inthis case, an indicator indicating a text portion corresponding to thevoice currently being reproduced is displayed on the display unit 151,and the indicator is moved in interlock with voice reproduction.

Furthermore, the mobile terminal 100 is implemented to enable thecontrol of a reproduction speed during voice reproduction, and theindicator is formed to change the movement speed or form thereofaccording to the reproduction speed. As an example of them, when theindicator is a block of highlighting a specific range as illustrated inFIG. 8J, it may be formed to highlight one word when the reproductionspeed of the block is 1× speed but highlight a plurality of words whenthe reproduction speed of the block is 2× speed.

As described above, according to a third embodiment of the presentdisclosure, the mobile terminal 100 may display a text string made ofcore keywords upon receiving an access request for a list of audiofiles, and may output the whole text when there is a gesture for thetext string from the user. Furthermore, according to a third embodimentof the present disclosure, the user may control the reproduction of anaudio file in a more convenient manner through a text string made of thecore keywords.

A fourth embodiment disclosed herein may be implemented by a part orcombination of configurations or steps included in the foregoingembodiments, or implemented by a combination of the embodiments.Hereinafter, the redundant portions will be omitted for the explicitexpression of the fourth embodiment disclosed herein.

FIG. 9A is a flow chart illustrating an audio file editing methodaccording to a fourth embodiment disclosed herein.

Referring to FIG. 9A, an audio file editing method according to a fourthembodiment disclosed herein may be carried out in the following steps.

First, the mobile terminal may reproduce a specific audio file among aplurality of audio files stored therein (S3011). Next, the mobileterminal may perform a real-time speech-to-text (STT) for the reproducedspecific audio file.

Furthermore, the mobile terminal may generate a text (text string orscript) corresponding to the audio file based on the performed real-timespeech-to-text (STT) (S3012).

Next, the mobile terminal may detect a space portion of the generatedtext (text string or script) (S3013). Here, the space portion may denotea portion with a vacant character (or vacant text string) contained inthe text (text string or script).

Next, the mobile terminal may delete an audio file portion correspondingto the detected space portion. Furthermore, the mobile terminal maygenerate a new audio file based on the remaining audio file portionexcluding the deleted audio file portion (S3014).

Referring to FIG. 9B, an audio file editing method according to a fourthembodiment disclosed herein may be applicable to a prestored file.

According to the editing method applicable to a prestored file, themobile terminal 100 may first select a specific audio file among aplurality of audio files stored therein (S3021). Next, the mobileterminal 100 may perform an speech-to-text (STT) for the selectedspecific audio file.

Furthermore, the mobile terminal 100 generates a text (text string orscript) corresponding to the audio file based on the performedspeech-to-text (STT) (S3022), and detects a space portion of thegenerated text (text string or script) (S3023).

Next, the mobile terminal deletes an audio file portion corresponding tothe detected space portion, and generates a new audio file based on theremaining audio file excluding the deleted audio file (S3024).

FIG. 9C is an exemplary view illustrating an audio file editing methodaccording to a fourth embodiment disclosed herein.

When the specific audio file (A001.wmv) contains a vacant audio section(A110), the user deletes the audio section (A110) that should be deletedbased on the audio file editing method according to a fourth embodimentdisclosed herein to generate a new audio file (A002.wmv).

The audio file editing method according to a fourth embodiment disclosedin FIG. 9C may be applicable both to a real-time speech-to-text (STT)while reproducing an audio file and to an speech-to-text (STT)subsequent to selecting an audio file.

Referring to FIG. 9C(a), the mobile terminal 100 may select a firstaudio file (A001.wmv) among audio files stored therein based on theterminal user's select input.

Furthermore, the mobile terminal 100 may reproduce the selected firstaudio file (A001.wmv). In this case, the mobile terminal 100 may performan speech-to-text (STT) for the selected first audio file (A001.wmv).

The mobile terminal 100 may display a first text (T110) generated basedon the speech-to-text (STT) in a partial region of the screen of themobile terminal.

Furthermore, the mobile terminal 100 may detect a space portion (E110)of the first text (T110).

In this case, the mobile terminal 100 may detect a portion (A110) of theaudio file (A001.wmv) corresponding to the detected space portion(E110).

Referring to FIG. 9C(b), the mobile terminal 100 may delete a portion(A110) of the audio file (A001.wmv) corresponding to the space portion(E110), and may generate a new audio file (A002.wmv) containing theremaining portion excluding the deleted portion (A110) of the audio file(A001.wmv).

In this case, it may be possible to check that the new audio file(A002.wmv) does not have the audio portion (A110) corresponding to thespace portion (E110) in FIG. 9C(b).

FIG. 9D is a flow chart illustrating another audio file editing methodaccording to a fourth embodiment disclosed herein.

Referring to FIG. 9D, an audio file editing method according to a fourthembodiment disclosed herein will be implemented in the following steps.

First, the mobile terminal may reproduce a specific audio file among aplurality of audio files stored therein (S3031). Next, the mobileterminal may perform a real-time speech-to-text (STT) for the reproducedspecific audio file.

Furthermore, the mobile terminal may generate a text (text string orscript) corresponding to the audio file based on the performed real-timespeech-to-text (STT) (S3032).

Next, the mobile terminal may detect part of the generated text (textstring or script) (S3033). Next, the mobile terminal may delete an audioportion corresponding to the selected text portion.

Furthermore, the mobile terminal may generate a new audio file based onthe remaining audio file portion excluding the deleted audio fileportion (S3034). However, the present disclosure is not limited to this,and the selected text portion may be generated and stored as a new audiofile.

As illustrated in FIG. 9E, an audio file editing method disclosed inFIG. 9D may be also applicable to a prestored file.

Referring to FIG. 9E, the mobile terminal first selects a specific audiofile among a plurality of audio files stored therein (S3041), andperforms an speech-to-text (STT) for the selected specific audio file.Hereinafter, the remaining process for generating a new audio file maybe carried out similarly to the method disclosed in FIG. 9D.

FIG. 9F is an exemplary view illustrating an audio file editing methoddisclosed in FIG. 9D or 9E. The audio file editing method disclosed inFIG. 9F may be applicable both to a real-time speech-to-text (STT) whilereproducing an audio file and to an speech-to-text (STT) subsequent toselecting an audio file.

Referring to FIG. 9F(a), the mobile terminal 100 may select a specificaudio file (B001.wmv) among audio files stored therein based on themobile terminal user's select input. The stored audio file may be atemporary storage file of audio currently being recorded.

When the specific audio file (B001.wmv) is selected, the mobile terminal100 may perform an speech-to-text (STT) for the specific audio file(B001.wmv).

The selected specific audio file (B001.wmv) may be reproduced or a textgenerated based on the speech-to-text (STT) may be displayed in apartial region of the screen of the mobile terminal. In this case, thereproduction and output of a text should be at the same time.

When the specific audio file (B001.wmv) contains an audio section (A210)that should be deleted, the audio section (A210) that should be deletedmay be deleted based on the audio file editing method to generate a newaudio file (B002.wmv).

More specifically, as illustrated in the drawing, the mobile terminal100 may select a specific text (T210) of the generated text based on theuser's select input.

The user's select input may be input to the mobile terminal in variousways. For example, as illustrated in FIG. 9F(a), the user may apply atouch input to the mobile terminal. Furthermore, the user may generate adrag event based on the touch input. In other words, the mobile terminal100 may select the specific text (T210) based on the drag. Here, it maybe a specific audio section (or portion, A210) of the audio file(B001.wmv) corresponding to the specific text (T210). In addition, itshould be understood by those skilled in the art that the user's selectinput can be applied to the mobile terminal in various ways.

Referring to FIG. 9F(b), the mobile terminal 100 may delete a specificportion (A210) of the audio file (B001.wmv) corresponding to thespecific text (T210), and may generate a new audio file (B002.wmv)containing the remaining portion excluding the deleted portion (A210) ofthe audio file (B001.wmv).

In this case, it may be possible to check that the new audio file(B002.wmv) does not have audio corresponding to the specific portion(A210) or the audio section (A210) corresponding to the specific text(T210) in FIG. 9F(b).

According to a modified fourth embodiment, on the contrary, the user mayselect a partial audio section of the audio file to separately store atext for the remaining audio section excluding the partial audiosection.

Furthermore, the mobile terminal 100 may display a reproduced audiosection indicator (for example, progress bar of FIG. 9F(a))corresponding to the audio file (B001.wmv) on the screen of the mobileterminal. Furthermore, the user may select the audio section (A210) onthe indicator based on a touch or drag.

FIG. 10A is a flow chart illustrating another embodiment for an audiofile is editing method according to the present disclosure.

Referring to FIG. 10A, the audio file editing method may select aspecific audio file among a plurality of audio files stored therein(S3051). Next, the mobile terminal may perform a real-timespeech-to-text (STT) for the selected audio file. In this case, thereproduction of the selected audio file may be operated at the sametime.

The mobile terminal may generate a text (text string or script)corresponding to the audio file based on the performed real-timespeech-to-text (STT) (S3052).

Next, a specific section of the generated text (text string or script)may be selected by the user (S3053). Finally, the mobile terminalprovide a priority to the selected specific section or generate a newaudio file corresponding to the specific section (S3054).

FIG. 10B is an exemplary view illustrating an audio file editing methodin FIG. 10A.

The audio file editing method disclosed in FIG. 10B may be applicableboth to a real-time speech-to-text (STT) while reproducing an audio fileand to an speech-to-text (STT) subsequent to selecting an audio file.

Referring to FIG. 10A, the mobile terminal 100 may select a specificaudio file (C001.wmv) among audio files stored therein based on theuser's select input.

By means of the selection, the mobile terminal 100 may display aterminal generated based on an speech-to-text (STT) in a partial regionof the screen of the mobile terminal. In this case, the mobile terminal100 may reproduce the selected specific audio file (C001.wmv) along withthe output of a text.

When it is desired to provide a priority to a partial audio section(A310) (or text section (T310)) in the audio section of the specificaudio file (C001.wmv), the user may generate a file (C002.wmv) providedwith a priority for the partial audio section (A310) based on the audiofile editing method in FIG. 10A. For example, when a continuous touch tothe text is sensed as illustrated in FIG. 10B(a), the mobile terminal100 selects an audio section using the former touch as a start point andthe latter touch as an end point. For another example, an audio sectionmay be selected by a drag.

A priority is provided to the audio section, and stored as a new file.However, the present disclosure is not necessarily limited to this, andthe priority may be provided to the initial audio file (C001.wmv) andupdated as a new file. When the user select a file (C002.wmv) providedwith a priority as illustrated in FIG. 10B(b), the output of a voice maybe first output to the audio section (A310) provided with a priority.

As described above, according to a fourth embodiment disclosed herein,the editing of an audio file may be easily implemented. Hereinafter, afifth embodiment disclosed herein will be described.

A fifth embodiment disclosed herein may be implemented by a part orcombination of configurations or steps included in the foregoingembodiments, or implemented by a combination of the embodiments.Hereinafter, the redundant portions will be omitted for the explicitexpression of the fifth embodiment disclosed herein.

A fifth embodiment disclosed herein illustrates a clustering method foran audio file (or voice memo, hereinafter, referred to as a “voicememo”) stored in the mobile terminal.

The concept of clustering may be a generic term for schemes forclassifying data into several groups based on according to a concept ofsimilarity or the like.

In general, the clustering may be widely applicable to documentretrieval, pattern recognition, management science, and the like.

According to a fifth embodiment disclosed herein, the user may select aclassification reference for the voice memos as a clustering method forvoice memos stored in the mobile terminal. The mobile terminal 100classifies and lists the voice memos based on the selectedclassification reference to generate a list for the voice memos, and thegenerated list for the voice memos on the screen of the mobile terminal.

Furthermore, a fifth embodiment disclosed herein relates to a method inwhich the mobile terminal analyzes voice content for the voice memos todisplay the associated information on the screen, thereby allowing theuser of the mobile terminal to easily send or transfer specific contentto a specific person. It may be called a direct access method through avoice memo (or audio file). Here, the associated information may includefor a pop-up window or link information on a place associated with thevoice content, a schedule, a name of person, and a contact address (forexample, phone number or email address).

FIG. 11A is a flow chart illustrating clustering (or classificationmethod) for a voice memo according to a fifth embodiment disclosedherein. Referring to FIG. 11A, clustering (or classification method) forthe voice memo may be carried out in the following steps.

First, the mobile terminal may display a UI screen associated with audiofile reproduction or recording stored in the mobile terminal (S3071).

Next, the mobile terminal may display a classification menu for theaudio is files on the UI screen based on the mobile terminal user'sselect input (S3072).

Next, the mobile terminal may select a specific classification referencefrom the classification menu based on the user's select input (S3073).

Next, the mobile terminal may display a list for the audio files on thescreen of the mobile terminal based on the selected classificationreference (S3074).

Here, the UI associated with audio file reproduction or recording mayinclude various UI elements. In other words, the UI may include variousUI elements associated with the audio file reproduction or recordingfunction by the mobile terminal.

For example, the UI (particularly, GUI) may include at least one of aplay button, a play suspend button, a function activation button capableof reproducing at high speed, a function activation button capable ofreproducing at low speed, a forward skip button, a backward skip button,an indicator indicating a reproduction audio section or reproductiontime point of the audio file, a voice recording button for the mobileterminal user, a recording suspend button, and a UI element associatedwith text display which is an speech-to-text (STT) execution result ofthe audio file.

FIG. 11B is an exemplary view illustrating clustering for a voice memoaccording to a fifth embodiment disclosed herein.

Referring to FIG. 11B, the mobile terminal 100 may display a UIassociated with the reproduction or recording of a voice memo storedtherein on the screen of the mobile terminal. As described above, the UImay include an indicator indicating clustering (or classification) forthe stored voice memos.

The indicator may be a button (U110) for activating a classificationmenu is for the clustering (or classification).

When the mobile terminal user selects the classification menu activationbutton, the mobile terminal 100 may display a classification menu (p110)for the stored voice memos on the screen of the mobile terminal.

The classification menu (p110) may include a UI element for variousclassification references about the store voice memos.

For example, the UI element for classification reference may be a selectbutton for the classification reference. Furthermore, the classificationreference may be a “subject” button (p111), a “speaker” button, a“place” button, or a “time” button for the voice memos.

According to a fifth embodiment, when the user of the mobile terminal100 selects a “subject” button (p111) among the classification buttons,the mobile terminal 100 may cluster (or classify) and list the voicememo for each “subject”, and display a list (L110) for the voice memoson the screen of the mobile terminal 100.

For example, as in case of FIG. 11B, when the voice memos have a subjectassociated with meeting, the mobile terminal 100 may list the voicememos for each subject of the meeting, and display a list (L110) for thevoice memos on the screen of the mobile terminal 100.

FIG. 11C is an exemplary view illustrating a direct access method basedon the content of a voice memo according to a fifth embodiment disclosedherein.

Referring to FIG. 11C, the mobile terminal 100 may display a UIassociated with the reproduction or recording of a voice memo storedtherein on the screen of the mobile terminal 100.

The mobile terminal 100 may select and reproduce a specific audio file(D001.wmv) among audio files stored in the mobile terminal 100 based onthe terminal user's select input through the UI.

In this case, the mobile terminal 100 may perform an speech-to-text(STT) for the specific audio file (D001.wmv), and may display a textgenerated based on the speech-to-text (STT) in a partial region of thescreen of the mobile terminal 100. Moreover, in the state of a textbeing displayed, it may be displayed an indicator indicating a portionof which a voice is currently reproduced as described above.

According to a fifth embodiment, the mobile terminal 100 may analyze thecontent of the voice memo to display information associated with thevoice memo on the screen of the mobile terminal 100.

For example, the information associated with the voice memo may includeat least one of a place at which the voice memo is recorded, a scheduleof person associated with the voice memo, a name of person associatedwith the voice memo, a contact address of person associated with thevoice memo, and an email address of person associated with the voicememo.

The mobile terminal 100 may display information associated with thevoice memo on the screen through a pop-up window (1110).

According to a fifth embodiment, when the user of the mobile terminal100 selects specific information contained in the pop-up window (1110)(for example, selection based on a touch input on the screen), themobile terminal 100 may provide a function capable of directly sendingor contacting the selected specific information to a person associatedwith the specific information based on the selected specificinformation. As described above, it may be referred to as a directaccess function based on information associated with the voice memo.

For example, referring to FIG. 11C, when the user selects a contactaddress portion among specific information contained in the pop-upwindow (1110), the mobile terminal 100 may perform a call connectionfunction with a phone number corresponding to the contact address.

Furthermore, for example, when the user selects an email portion amongspecific information contained in the pop-up window (1110), the mobileterminal 100 may execute an application (for example, email editingprogram, Outlook program) for providing an email transfer function totransfer an email to the email address.

A sixth embodiment disclosed herein may be implemented by a part orcombination of configurations or steps included in the foregoingembodiments, or implemented by a combination of the embodiments.Hereinafter, the redundant portions will be omitted for the explicitexpression of the sixth embodiment disclosed herein.

A sixth embodiment disclosed herein relates to a search function for aspecific audio section contained in a voice memo stored in the mobileterminal. The search function may be a navigation function for aspecific audio section contained in the voice memo. Otherwise, thesearch function may be also a voice scanning function for a specificaudio section contained in the voice memo.

Furthermore, a sixth embodiment disclosed herein relates to a search,navigation or scanning function for a specific text sectioncorresponding to the specific audio section of a text which is anspeech-to-text (STT) execution result corresponding to the voice memoaccording to the search function, the navigation function, or the voicescanning function.

As a result, according to a sixth embodiment disclosed herein, the voicememo and the search function for a text which is an speech-to-text (STT)execution result corresponding to the voice memo may have a reversiblerelation with each other.

In other words, when a search function for the voice memo is carriedout, a search function for a text with respect to the voice memo may bealso carried out at the same time. Furthermore, when a search functionfor a text with respect to the voice memo is carried out, a searchfunction for the voice memo may be also carried out at the same time.

FIG. 12A is a flow chat illustrating a search function execution methodfor a voice memo according to a sixth embodiment disclosed herein.Referring to FIG. 12A, a search function execution method for the voicememo may be carried out in the following steps.

First, the mobile terminal may reproduce a specific audio file among aplurality of audio files stored in the electronic device (S3081).

Next, the mobile terminal may perform a real-time speech-to-text (STT)for the reproduced specific audio file, and generate a text (text stringor script) corresponding to audio file based on the performed real-timespeech-to-text (STT) (S3082). However, in this case, the step ofgenerating a text (S3082) may be omitted when a text based on STT hasbeen generated or when a text has been generated based on other methods.

Next, the mobile terminal may select a specific reproduction time pointfor the audio file based on the mobile terminal user's select input(S3083). To this end, the mobile terminal may display a text portioncorresponding to the specific reproduction time point of the text (textstring or script) on the screen of the mobile terminal (S3084).

FIG. 12B is an exemplary view illustrating a search function executionmethod for a voice memo according to a sixth embodiment disclosedherein.

The mobile terminal 100 may display a UI associated with thereproduction or recording of a voice memo (or audio file) on the screenof the mobile terminal 100.

The mobile terminal may select and reproduce a specific audio file(E001.wmv) among audio files stored in the mobile terminal based on themobile terminal user's select input. At this time, the mobile terminalmay display a text corresponding to a voice in a partial region of thescreen of the mobile terminal.

According to a fifth embodiment, when the user of the mobile terminalselects a specific reproduction time point (p210) in the audio sectionof the specific audio file (E001.wmv), the mobile terminal 100 may find(or search) a portion corresponding to the specific reproduction timepoint (p210) in the text to display it on the screen of the mobileterminal.

When a search function for the text is described in detail withreference to FIG. 12B(a), the mobile terminal 100 may display areproduced audio section indicator (for example, an audio reproductionsection bar in FIG. 10C(a)) corresponding to the audio file (E001.wmv)on the screen of the mobile terminal.

Furthermore, the mobile terminal 100 may select the specificreproduction time point (p210) on the indicator based on the user'stouch input. Moreover, the text displayed during the search using theprogress bar operated by the user and the progress bar may be linkedwith each other.

For such an example, the mobile terminal 100 may reproduce the audiofile (E001.wmv) from the specific reproduction time point (p210), anddisplay a specific text (T410) corresponding to the selected specificreproduction time point (p210) in a text for the audio file (E001.wmv)at the same time on the screen of the mobile terminal 100.

More specifically, the controller enters an operation mode associatedwith the reproduction of a voice memo stored according to a user'srequest, and displays a text corresponding to a reproduction time pointof the voice memo in at least a partial region of the screen of theelectronic device in the operation mode, and senses a control commandfor changing the reproduction time point of the voice memo to change andoutput the displayed text to correspond to the changed reproduction timepoint when the control command is sensed.

A progress bar indicating the time information of the voice memo isdisplayed on the screen of the electronic device, and a reproductiontime point for the voice memo is changed to correspond to a touch pointfor the progress bar.

In this case, the text is generated based on a speech-to-text (STT)conversion for the user's voice, and the text may be displayed whilebeing scrolled in linkage with the reproduction of the voice memo.

For still another example, referring to FIG. 12B(b), a separate searchwindow may be popped up by the user's control command input. For such anexample, the search window may be carried out to receive a word desiredto search as illustrated in the drawing, or a progress bar may bedisplayed as a search bar as illustrated in FIG. 12A. In case of theformer, the user may input a word to perform a search operation, and thesearch result may be displayed in such a manner that sections used bythe input word can be selected again by the user. In case of the latter,the user touches a specific point of the progress bar to perform asearch operation, and as illustrated in the foregoing example, a text(or sentence containing the text) corresponding to the specific point isdisplayed on the screen when making a touch.

FIG. 12C is a flow chart illustrating a search function execution methodfor a voice memo according to a sixth embodiment disclosed herein.

Referring to FIG. 12C, a search function execution method for a voicememo (or audio file) according to a sixth embodiment disclosed hereinmay be carried out in the following steps.

The steps of S3091 and S3092 in the present example may be carried outsimilarly to the steps of S3081 and S3082 in a search function executionmethod illustrated in FIG. 12A.

According to the drawing, subsequent to the step S3092, the mobileterminal selects a specific text portion in the text (text string orscript) based on the mobile terminal user's select input (S3093), andreproduces an audio file portion corresponding to the specific textportion (S3084).

However, the present disclosure is not necessarily limited to this. Forexample, when storing the audio file subsequent to the execution of STT,a new text file may be also stored along with the audio file. Throughthis, a text corresponding to the reproduced voice can be immediatelydisplayed using a text file without a separate STT process during thereproduction of the audio file later. For such an operation, a timecorresponding to a specific portion of the text may be synchronized witha time of the portion corresponding to the specific portion in the audiofile.

FIG. 12D is an exemplary view illustrating a search function executionmethod for a voice memo according to a sixth embodiment disclosedherein.

Referring to FIG. 12D(a), the mobile terminal 100 may display a UIassociated with the reproduction or recording of a voice memo (or audiofile) stored therein on the screen of the mobile terminal 100.

The mobile terminal 100 selects and reproduces a specific audio file(F001.wmv) among audio files stored therein based on the mobile terminaluser's select input. Furthermore, a text (T510) generated based on anspeech-to-text (STT) may be displayed in a partial region of the screenof the mobile terminal.

The mobile terminal 100 may select a specific text (T520) from thegenerated text (T510) based on the user's select input.

The selection for the specific text (T520) may be carried out based onthe user's touch input and a drag event on the basis of the touch input.The mobile terminal 100 may select the specific text (T520) based on thedrag event.

Referring to FIG. 12D(b), when the specific text (T520) is selected, themobile terminal 100 may change the reproduction time point of the audiofile (F001.wmv) from a reproduction time point (p310) of the audio file(F001.wmv) at present to a specific reproduction time point (p320)corresponding to the specific text (T520).

At this time, the mobile terminal 100 may display a text (T530)corresponding to the specific reproduction time point in an upper endportion of the screen of the mobile terminal while at the same timechanging the reproduction time point of the audio file (F001.wmv).

Referring to FIG. 12E, for another example, when the user touches aspecific position of the progress bar while reproducing voice, a text ata specific time point corresponding to the specific position may bedisplayed along with the indicator. Then, when the user touches the textor indicator, it may be possible to change a reproduction time point ofthe audio file (G001.wmv) to correspond to the portion as well asdisplay a text corresponding to the specific reproduction time point ina partial region of the screen of the mobile terminal.

FIG. 12F is an exemplary view illustrating an audio scanning methodaccording to a sixth embodiment disclosed herein.

The sixth embodiment disclosed in FIG. 12F illustrates a case in whichthe mobile terminal displays a specific text portion corresponding tothe reproduction time point of the audio file in a text based on theexecution of an speech-to-text (STT) to the audio file according to aspecific effect along with the reproduction of the audio file storedtherein.

For example, the specific effect may be an effect of displaying thespecific text portion while moving it from the right side of the screento the left side thereof. Furthermore, the movement of the specific textmay be expressed along with a drift effect.

Furthermore, for example, the specific effect may refer to a highlighteffect of the specific text portion. In this case, the highlight effectmay refer to an effect of enhancing the specific text portion (forexample, a bold character or relatively larger sized character comparedto other text portions).

Furthermore, an audio scanning method according to a sixth embodimentdisclosed in FIG. 12F may include a function in which when areproduction time point of the audio file is changed by the mobileterminal user's select input during the reproduction of the audio filestored in the mobile terminal, it is changed and displayed with a textportion corresponding to the changed reproduction time point.

Referring to FIG. 12F(a), the mobile terminal 100 may reproduce theaudio file at a first reproduction time point (p410).

In this case, the mobile terminal 100 may display a text (T610) (“DaehoLee . . . on the thirtieth”) corresponding to the first reproductiontime point on the is screen of the mobile terminal.

Referring to FIG. 12F(b), when the mobile terminal user changes areproduction time point for the audio file to a second reproduction timepoint (p420), the mobile terminal 100 may display a text (T620) (“ . . .a solo homer”) corresponding to the second reproduction time point(p420) along with a change of the reproduction time point for the audiofile on the screen.

FIG. 12G is an exemplary view illustrating a language information changemethod for an audio file according to another sixth embodiment disclosedherein.

Referring to FIG. 12G, the mobile terminal 100 may change a specificlanguage voice to another language voice to perform voice recording.

For example, when the mobile terminal user utters “Naneun sonyeonida” inKorean voice while performing voice recording by the mobile terminal,the mobile terminal 100 may record the Korean voice as it is ortranslate the Korean voice to record it with English voice (for example,“I am a boy”).

In this case, the mobile terminal 100 may output the English voice (forexample, “I am a boy”) with a voice through an audio output device suchas a speaker or the like along with the storage of the English voice.

Furthermore, the mobile terminal 100 may perform an speech-to-text (STT)for the English voice to generate a text for the English voice (forexample, “I am a boy”).

Furthermore, the mobile terminal 100 may display a text for thegenerated English voice (for example, “I am a boy”) on the screen of themobile terminal.

A seventh embodiment disclosed herein may be implemented by a part orcombination of configurations or steps included in the foregoingembodiments, or implemented by a combination of the embodiments.Hereinafter, the redundant is portions will be omitted for the explicitexpression of the seventh embodiment disclosed herein.

A seventh embodiment disclosed herein relates to an automaticreproduction function of a voice memo (or audio file) stored in themobile terminal, and is related to activating an automatic reproductionfunction of the voice memo (or audio file) when a part of the user'sbody (for example, the user's ear) is located proximate to the mobileterminal.

FIG. 13A is a flow chart illustrating an automatic reproduction methodof an audio file according to a seventh embodiment disclosed herein, andFIG. 13B is an exemplary view illustrating an automatic reproductionmethod of an audio file according to a seventh embodiment disclosedherein.

Referring to FIG. 13A, an automatic reproduction method of an audio fileaccording to a seventh embodiment disclosed herein may be carried out inthe following steps.

First, the mobile terminal may select a specific audio file among aplurality of audio files stored therein (S3111). Next, the mobileterminal may reproduce the specific audio file when a part of the user'sbody is located proximate to the mobile terminal (S3112).

Referring to FIG. 13B, the mobile terminal 100 may display a lest foraudio files stored in the mobile terminal on the screen of the mobileterminal.

In this case, the user of the mobile terminal may select a specificaudio file (C001.wmv) from a list for the audio files. The selection ofthe audio file (C001.wmv) may be based on a touch input by the user.

Referring to FIG. 13C, when a part of the user's body (for example, theuser's ear) is located proximate to the mobile terminal, the mobileterminal 100 may automatically reproduce the specific audio file(C001.wmv).

In this manner, according to the main feature of the present disclosure,it allows the user to easily write a specific time point during theaudio recording. Accordingly, the facilitation of retrieval can beprovided later.

As described above, according to the embodiments of the presentdisclosure, the electronic device, for instance, mobile terminal 100,may perform an speech-to-text (STT) or algorithm to acquire a text, textstring or script, and then display a file name or text string made ofcore keywords. Through this, the user may easily guess the content ofthe audio file without reproducing the audio file.

Furthermore, according to the present disclosure, when there is agesture for the text string or file name from the user, the mobileterminal may display the whole text. Through this, extraction for voicecontent may be carried out through the text.

Furthermore, according to the present disclosure, the user may controlthe reproduction of an audio file in a more convenient manner throughthe core keywords or a text string made thereof.

In the above, the entire process associated with an audio file has beendescribed in detail by dividing it into embodiments. Hereinafter, thehardware configuration of a mobile terminal for implementing theforegoing function will be described, and moreover, the hardwareconfiguration of an image display device and a refrigerator amongelectronic devices to which the foregoing function is applicable.

FIG. 14 is a block diagram illustrating a mobile terminal forimplementing the foregoing function.

The mobile terminal 100 may include a wireless communication unit 110,an audio/video (A/V) input unit 120, a user input unit 130, a sensingunit 140, an output unit 150, a memory 160, an interface unit 170, acontroller 180, a power supply unit 190, and the like. However, theconstituent elements as illustrated in FIG. 14 are not necessarilyrequired, and the mobile terminal may be implemented with greater orless number of elements than those illustrated elements.

Hereinafter, the constituent elements will be described in sequence.

The wireless communication unit 110 typically includes one or moreelements allowing radio communication between the mobile terminal 100and a wireless communication system, or allowing radio communicationbetween radio communication the mobile terminal 100 and a network inwhich the mobile terminal 100 is located. For example, the wirelesscommunication unit 110 may include a broadcast receiving module 111, amobile communication module 112, a wireless Internet module 113, ashort-range communication module 114, a location information module 115,and the like.

The broadcast receiving module 111 receives broadcast signals and/orbroadcast associated information from an external broadcast managementserver through a broadcast channel.

The broadcast channel may include a satellite channel and/or aterrestrial channel. The broadcast management server may mean a serverthat generates and transmits a broadcast signal and/or broadcastassociated information or a server that receives a previously generatedbroadcast signal and/or broadcast associated information and transmitsto the mobile terminal 100. The broadcast signal may include a TVbroadcast signal, a radio broadcast signal and a data broadcast signalas well as a broadcast signal in a form that a data broadcast signal iscoupled to the TV or radio broadcast signal.

The broadcast associated information may mean information regarding abroadcast channel, a broadcast program, a broadcast service provider,and the like. The broadcast associated information may also be providedthrough a mobile communication network, and in this case, the broadcastassociated information may be received by the mobile communicationmodule 112.

The broadcast associated information may exist in various forms. Forexample, it may exist in the form of an electronic program guide (EPG)of digital multimedia broadcasting (DMB), electronic service guide (ESG)of digital video broadcast-handheld (DVB-H), and the like.

The broadcast receiving module 111 may receive a broadcast signal usingvarious types of broadcast systems. In particular, the broadcastreceiving module 111 may receive a digital broadcast signal using adigital broadcast system such as digital multimediabroadcasting-terrestrial (DMB-T), digital multimediabroadcasting-satellite (DMB-S), media forward link only (MediaFLO),digital video broadcast-handheld (DVB-H), integrated services digitalbroadcast-terrestrial (ISDB-T), and the like. The broadcast receivingmodule 111 is, of course, configured to be suitable for every broadcastsystem that provides a broadcast signal as well as the above-mentioneddigital broadcast systems.

The broadcast signal and/or broadcast-associated information receivedthrough the broadcast receiving module 111 may be stored in the memory160.

The mobile communication module 112 transmits and/or receives a radiosignal to and/or from at least one of a base station, an externalterminal and a server over a mobile communication network. Here, theradio signal may include a voice call signal, a video call signal and/orvarious types of data according to text and/or multimedia messagetransmission and/or reception.

The wireless Internet module 113 means a module for supporting wirelessInternet access. The wireless Internet module 113 may be built-in orexternally installed to the mobile terminal 100. Here, it may be used awireless Internet access technique including a WLAN (Wireless LAN),Wi-Fi, Wibro (Wireless Broadband), Wimax (World Interoperability forMicrowave Access), HSDPA (High Speed Downlink Packet Access), and thelike.

The short-range communication module 114 is a module for supporting ashort-range communication. Here, it may be used a short-rangecommunication technology including Bluetooth, Radio FrequencyIDentification (RFID), Infrared Data Association (IrDA), Ultra WideBand(UWB), ZigBee, and the like.

The location information module 115 is a module for acquiring thelocation of a mobile terminal, and there is a Global Positioning System(GPS) module or Wireless Fidelity (WiFi) module as a representativeexample.

Referring to FIG. 14, the A/V (audio/video) input unit 120 receives anaudio or video signal, and the A/V (audio/video) input unit 120 mayinclude a camera 121 and a microphone 122. The camera 121 processes aimage frame, such as still picture or video, obtained by an image sensorin a video phone call or image capturing mode. The processed image framemay be displayed on a display unit 151.

The image frames processed by the camera 121 may be stored in the memory160 or transmitted to an external device through the wirelesscommunication unit 110. Two or more cameras 121 may be providedaccording to the use environment of the mobile terminal.

The microphone 122 receives an external audio signal through amicrophone in a phone call mode, a recording mode, a voice recognitionmode, and the like, and processes the audio signal into electrical voicedata. The processed voice data may be converted and outputted into aformat that is transmittable to a mobile communication base stationthrough the mobile communication module 112 in the phone call mode. Themicrophone 122 may implement various types of noise cancellingalgorithms to cancel noise generated in a procedure of receiving theexternal audio signal.

Moreover, the A/V (audio/video) input unit 120 may be used even foraudio recording. The A/V (audio/video) input unit 120 may be linked withthe recording function unit 123 and conversion unit 124 to implement thefunction of audio recording.

The recording function unit 123 performs the function of recording anexternal sound to store it as an audio file, and the conversion unit 124performs the function of converting a voice contained in the sound intoa text based on a speech-to-text (STT) conversion. The controller 180may detect a core keyword from the text, and set the detected corekeyword to at least part of a file name for the audio file, and thus thefile name of the audio file may be easily set up.

The user input unit 130 may generate input data to control an operationof the terminal. The user input unit 130 may be configured by includinga keypad, a dome switch, a touch pad (pressure/capacitance), a jogwheel, a jog switch, and the like.

The sensing unit 140 detects a current status of the mobile terminal 100such as an opened or closed state of the mobile terminal 100, a locationof the mobile terminal 100, an orientation of the mobile terminal 100,and the like, and generates a sensing signal for controlling theoperation of the mobile terminal 100. For example, when the mobileterminal 100 is a slide phone type, it may sense an opened or closedstate of the slide phone. Furthermore, the sensing unit 140 takes chargeof a sensing function associated with whether or not power is suppliedfrom the power supply unit 190, or whether or not an external device iscoupled to the interface unit 170.

The output unit 150 is configured to provide an output for audio signal,video signal, or alarm signal, and the output unit 150 may include thedisplay unit 151, an audio output module 152, an alarm unit 153, ahaptic module 154, and the like.

The display unit 151 may display (output) information processed in themobile terminal 100. For example, when the mobile terminal 100 is in aphone call mode, the display unit 151 may display a User Interface (UI)or a Graphic User Interface (GUI) associated with a call. When themobile terminal 100 is in a video call mode or image capturing mode, thedisplay unit 151 may display a captured image and/or received image, aUI or GUI.

The display unit 151 may include at least one of a Liquid CrystalDisplay (LCD), a Thin Film Transistor-LCD (TFT-LCD), an Organic LightEmitting Diode (OLED) display, a flexible display, a three-dimensional(3D) display, and an e-ink display.

Some of those displays may be configured with a transparent or opticaltransparent type to allow viewing of the exterior through the displayunit, which may be called transparent displays. An example of thetypical transparent displays may include a transparent LCD (TOLED), andthe like. Under this configuration, a user can view an object positionedat a rear side of a terminal body through a region occupied by thedisplay unit 151 of the terminal body.

Two or more display units 151 may be implemented according to aconfigured aspect of the mobile terminal 100. For instance, a pluralityof the display units 151 may be arranged on one surface to be spacedapart from or integrated with each other, or may be arranged ondifferent surfaces.

When the display unit 151 and a touch sensitive sensor (hereinafter,referred to as a “touch sensor”) have an interlayer structure(hereinafter, referred to as a “touch screen”), the display unit 151 maybe used as an input device rather than an output device. The touchsensor may be implemented as a touch film, a touch sheet, a touch pad,and the like.

The touch sensor may be configured to convert changes of a pressureapplied to a specific part of the display unit 151, or a capacitanceoccurring from a specific part of the display unit 151, into electricinput signals. Also, the touch sensor may be configured to sense notonly a touched position and a touched area, but also a touch pressure.

When there is a touch input to the touch sensor, the correspondingsignals are transmitted to a touch controller (not shown). The touchcontroller processes the received signals, and then transmitscorresponding data to the controller 180. Accordingly, the controller180 may sense which region of the display unit 151 has been touched.

Referring to FIG. 14, a proximity sensor 141 may be arranged at an innerregion of the mobile terminal 100 covered by the touch screen, or nearthe touch screen. The proximity sensor indicates a sensor to sensepresence or absence of an object approaching to a surface to be sensed,or an object disposed near a surface to be sensed, by using anelectromagnetic field or infrared rays without a mechanical contact. Theproximity sensor has a longer lifespan and a more enhanced utility thana contact sensor.

The examples of the proximity sensor may include an optical transmissiontype photoelectric sensor, a direct reflective type photoelectricsensor, a mirror reflective type photoelectric sensor, a high-frequencyoscillation proximity sensor, a capacitance type proximity sensor, amagnetic type proximity sensor, an infrared rays proximity sensor, andso on. When the touch screen is implemented as a capacitance type,proximity of a pointer to the touch screen is sensed by changes of anelectromagnetic field. In this case, the touch screen (touch sensor) maybe categorized into a proximity sensor.

Hereinafter, for the sake of convenience of brief explanation, a statusthat the pointer is positioned to be proximate onto the touch screenwithout contact will be referred to as “proximity touch”, whereas astatus that the pointer substantially comes in contact with the touchscreen will be referred to as “contact touch”. For the positioncorresponding to the proximity touch of the pointer on the touch screen,such position corresponds to a position where the pointer facesperpendicular to the touch screen upon the proximity touch of thepointer.

The proximity sensor senses proximity touch, and proximity touchpatterns (e.g., distance, direction, speed, time, position, movingstatus, etc.). Information relating to the sensed proximity touch andthe sensed proximity touch patterns may be output onto the touch screen.

The audio output module 152 may output audio data received from thewireless communication unit 110 or stored in the memory 160, in acall-receiving mode, a call-placing mode, a recording mode, a voicerecognition mode, a broadcast reception mode, and so on. The audiooutput module 152 may output audio signals relating to functionsperformed in the mobile terminal 100, e.g., sound alarming a callreceived or a message received, and so on. The audio output module 152may include a receiver, a speaker, a buzzer, and so on.

The alarm 153 outputs signals notifying occurrence of events from themobile terminal 100. The events occurring from the mobile terminal 100may include call received, message received, key signal input, touchinput, and so on. The alarm 153 may output not only video or audiosignals, but also other types of signals such as signals notifyingoccurrence of events in a vibration manner. Since the video or audiosignals can be output through the display unit 151 or the audio outputunit 152, the display unit 151 and the audio output module 152 may becategorized into a part of the alarm 153.

The haptic module 154 generates various tactile effects which a user canfeel. A representative example of the tactile effects generated by thehaptic module 154 includes vibration. Vibration generated by the hapticmodule 154 may have a controllable intensity, a controllable pattern,and so on. For instance, different vibration may be output in asynthesized manner or in a sequential manner.

The haptic module 154 may generate various tactile effects, includingnot only vibration, but also arrangement of pins vertically moving withrespect to a skin being touched, air injection force or air suctionforce through an injection hole or a suction hole, touch by a skinsurface, presence or absence of contact with an electrode, effects bystimulus such as an electrostatic force, reproduction of cold or hotfeeling using a heat absorbing device or a heat emitting device, and thelike.

The haptic module 154 may be configured to transmit tactile effectsthrough a user's direct contact, or a user's muscular sense using afinger or a hand. The haptic module 154 may be implemented in two ormore in number according to the configuration of the mobile terminal100.

The memory 160 (or storage unit) may store a program for processing andcontrolling the controller 180. Alternatively, the memory 160 maytemporarily store input/output data (e.g., phonebook data, messages,audios, still images, videos, and the like). Also, the memory 160 maystore data related to various patterns of vibrations and soundsoutputted upon the touch input on the touch screen.

The memory 160 may be implemented using any type of suitable storagemedium including a flash memory type, a hard disk type, a multimediacard micro type, a memory card type (e.g., SD or DX memory), RandomAccess Memory (RAM), Static Random Access Memory (SRAM), Read-OnlyMemory (ROM), Electrically Erasable Programmable Read-only Memory(EEPROM), Programmable Read-only Memory (PROM), magnetic memory,magnetic disk, optical disk, and the like. Also, the mobile terminal 100may operate a web storage which performs the storage function of thememory 160 on the Internet.

Furthermore, an application for performing a specific function may bestored in the memory 160. Moreover, an audio file generated by recordingmay be stored in the memory 160.

The interface unit 170 may generally be implemented to interface themobile terminal with external devices. The interface unit 170 may allowa data reception from an external device, a power delivery to eachcomponent in the mobile terminal 100, or a data transmission from themobile terminal 100 to an external device. The interface unit 170 mayinclude, for example, wired/wireless headset ports, external chargerports, wired/wireless data ports, memory card ports, ports for couplingdevices having an identification module, audio Input/Output (I/O) ports,video I/O ports, earphone ports, and the like.

The identification module may be configured as a chip for storingvarious information required to authenticate an authority to use themobile terminal 100, which may include a User Identity Module (UIM), aSubscriber Identity Module (SIM), and the like. Also, the device havingthe identification module (hereinafter, referred to as “identificationdevice”) may be implemented in a type of smart card. Hence, theidentification device can be coupled to the mobile terminal 100 via aport.

The interface unit may serve as a path for power to be supplied from anexternal cradle to the mobile terminal 100 when the mobile terminal 100is connected to the external cradle or as a path for transferringvarious command signals inputted from the cradle by a user to the mobileterminal 100. Such various command signals or power inputted from thecradle may operate as signals for recognizing that the mobile terminalhas accurately been mounted to the cradle.

The controller 180 typically controls the overall operations of themobile terminal 100. For example, the controller 180 performs thecontrol and processing associated with telephony calls, datacommunications, video calls, and the like. Furthermore, the controller180 may perform the operation of the first through the seventhembodiment disclosed herein.

The controller 180 may include a multimedia module 181 for reproducingmultimedia data. The multimedia module 181 may be implemented in anintegrated manner within the controller 180 or may be implemented in aseparate manner from the controller 180.

Furthermore, the controller 180 can perform a pattern recognitionprocessing so as to recognize writing or drawing input on the touchscreen as text or image.

The power supply unit 190 receives external power and internal powerunder the control of the controller 180 to provide power required byvarious components.

Various embodiments described herein may be implemented in a medium thatcan be read by a computer or similar device using software, hardware, orany combination thereof.

For hardware implementation, it may be implemented by using at least oneof application specific integrated circuits (ASICs), digital signalprocessors (DSPs), digital signal processing devices (DSPDs),programmable logic devices (PLDs), field programmable gate arrays(FPGAs), processors, controllers, micro-controllers, microprocessors,electrical units designed to perform the functions described herein. Insome cases, such embodiments may be implemented in the controller 180itself.

For software implementation, the embodiments such as procedures orfunctions may be implemented together with separate software modules.The software modules may perform at least one function or operationdescribed herein. Software codes can be implemented by a softwareapplication written in any suitable programming language. The softwarecodes may be stored in the memory 160 and executed by the controller180.

On the other hand, the function of the foregoing mobile terminal 100according to the embodiments of the present disclosure may beimplemented in an image display device which will be described later.Hereinafter, an image display device having an operation controlfunction of the mobile terminal 100 according to the embodiments of thepresent disclosure will be described with reference to FIG. 15.

FIG. 15 is an internal block diagram illustrating an image displaydevice according to an embodiment of the present disclosure.

Referring to FIG. 15, the image display device 200 according to anembodiment of the present disclosure may include a broadcast receiver205, an external device interface unit 235, a storage unit 240, a userinput interface unit 250, a controller 270, a display unit 280, an audiooutput unit 285, and a power supply unit 290. Among them, the broadcastreceiver 205 may include a tuner 210, a demodulation unit 220, and anetwork interface unit 230. Among them, the tuner 210 and thedemodulation unit 220 may be alternatively provided for the networkinterface unit 230.

The tuner 210 selects a RF broadcast signal corresponding to the channelselected by the user or every prestored channel from the radio frequency(RF) broadcast signals received through an antenna. Furthermore, thetuner 210 transforms the selected RF broadcast signal into anintermediate frequency signal, a baseband image, or an audio signal.

For example, the selected RF broadcast signal may be transformed into adigital IF (DIF) signal if it is a digital broadcast signal, and may betransformed into an analog baseband video or audio signal (CVBS/SIF) ifit is an analog broadcast signal. In other words, the tuner 210 canprocess both digital broadcast signals and analog broadcast signals. Theanalog baseband video or audio signal (CVBS/SIF) output from the tuner210 may be directly input to the controller 270.

Furthermore, the tuner 210 may receive RF broadcast signals with asingle carrier according to the Advanced Television System Committee(ATSC) method or RF broadcast signals having with a plurality ofcarriers according to the Digital Video Broadcasting (DVB) method.

On the other hand, the tuner 210 may sequentially select RF broadcastsignals on all broadcast channels that have been stored through achannel storage function among the RF broadcast signals received throughthe antenna to transform it to an intermediate frequency signal orbaseband video or audio signal.

The demodulation unit 220 receives a digital IF (DIF) signal that hasbeen transformed by the tuner 210 to perform a demodulation operation.

For example, if the digital IF signal output from the tuner 210 is theATSC method, then the demodulation unit 220 may perform 8-vestigal sideband (8-VSB) demodulation, for instance. Furthermore, the demodulationunit 220 may perform channel decoding. For this purpose, thedemodulation unit 220 may include a Trellis decoder, a de-interleaver, aReed-Solomon decoder, and the like, to perform Trellis decoding,de-interleaving, and Reed-Solomon decoding.

For example, if the digital IF signal output from the tuner 210 is theDVB method, then the demodulation unit 220 may perform Coded OrthogonalFrequency Division Modulation (COFDMA) demodulation, for instance.Furthermore, the demodulation unit 220 may perform channel decoding. Forthis purpose, the demodulation unit 220 may include a convolutiondecoder, a de-interleaver, a Reed-Solomon decoder and the like toperform convolution decoding, de-interleaving, and Reed-Solomondecoding.

The demodulation unit 220 may perform demodulation and channel decodingand then output a stream signal (TS). Here, the stream signal may be amultiplexed signal with video, audio, or data signals. For example, thestream signal may be a multiplexed MPEG-2 Transport Stream (TS) with anMPEG-2 video signal, a Dolby AC-3 audio signal, and the like. Morespecifically, MPEG-2 TS may include a 4-byte header, and a 184-bytepayload.

On the other hand, the foregoing demodulation unit 220 may be providedin a separate manner according to the ATSC method or DVB method. Inother words, it can be provided with an ATSC demodulation unit and a DVBdemodulation unit.

The stream signal output from the demodulation unit 220 may be input tothe controller 270. The controller 270 may perform inverse-multiplexing,video/audio signal processing and the like, and then output video to thedisplay unit 280, and output audio to the audio output unit 285.

The external device interface unit 235 may be provided to connect anexternal device with the image display device 200. For this purpose, theexternal device interface unit 235 may include an A/V input and outputunit (not shown) or wireless communication unit (not shown).

The external device interface unit 235 may be connected to an externaldevice such as a digital versatile disc (DVD), a Blu-ray disc, a gamingdevice, a camera, a camcorder, a computer (notebook) and the like in awired/wireless manner. The external device interface unit 235 maytransfer video, audio or data signals received from the outside throughan external device connected thereto to the controller 270 of the imagedisplay device 200. Furthermore, the external device interface unit 235may output video, audio or data signals processed by the controller 270to the external device connected thereto. For this purpose, the externaldevice interface unit 235 may include an A/V input and output unit (notshown) or wireless communication unit (not shown).

The A/V input and output unit may include a USB terminal, a CompositeVideo Banking Sync (CVBS) terminal, a component terminal, a S-videoterminal (analog), a Digital Visual Interface (DVI) terminal, a HighDefinition Multimedia Interface (HDMI) terminal, a RGB terminal, a D-SUBterminal, and the like to input video and audio signals of the externaldevice to the image display device 200.

The wireless communication unit may perform short-range wirelesscommunication with other electronic devices. The image display device200 may be connected to other electronic devices in a network accordingto a communication standard such as Bluetooth, Radio FrequencyIdentification (RFID), Infrared Data Association (IrDA), Ultra Wideband(UWB), ZigBee, Digital Living Network Alliance (DLNA), and the like.

Furthermore, the external device interface unit 235 may be connected toat least one of various set-top boxes and the foregoing variousterminals to perform an input and output operation with the set-top box.

On the other hand, the external device interface unit 235 may receive anapplication or application list within the adjoining external device totransfer it to the controller 270 or the storage unit 240.

The network interface unit 230 provides an interface for connecting theimage display device 200 to a wired/wireless network including theInternet network. The network interface unit 230 may include an Ethernetterminal, or the like, for example, for the connection with a wirednetwork, and a communication standard such as Wireless LAN (WLAN,Wi-Fi), Wireless broadband (Wibro), World Interoperability for MicrowaveAccess (Wimax), High Speed Downlink Packet Access (HSDPA), for example,for the connection with a wireless network.

The network interface unit 230 may be connected to a specific web pageis through a network. In other words, the network interface unit 230 maybe connected to a specific web page through a network to send or receivedata to or from the relevant server. In addition, the network interfaceunit 230 may receive content or data provided by the content provider ornetwork operator. In other words, the network interface unit 230 mayreceive content and information related to the content such as a movie,an advertisement, a game, VOD, a broadcast signal and the like, providedfrom the content provider or network provider through a network.Furthermore, the network interface unit 230 may receive the firmware'supdate information or update file provided by the network operator.Furthermore, the network interface unit 230 may send data to theInternet, content provider, or network operator.

Furthermore, the network interface unit 230 may receive a desiredapplication among the applications open to the public through a network.

The storage unit 240 may store programs for each signal processing orcontrol within the controller 270 and may store signal-processed video,audio, or data signals.

Furthermore, the storage unit 240 may perform a function for temporarilystoring video, audio, or data signals received from the external deviceinterface unit 235 or network interface unit 230. Furthermore, thestorage unit 240 may store information for a predetermined broadcastchannel through a channel storage function.

Furthermore, the storage unit 240 may store an application orapplication list received from the external device interface unit 235 ornetwork interface unit 230.

The storage unit 240 may include at least one type of storage mediumincluding a flash memory type, a hard disk type, a multimedia card microtype, a card-type memory (e.g., SD or XD memory, etc), a Random AccessMemory (RAM), a Read-Only Memory (EPROM, etc.), and the like. The imagedisplay device 200 may reproduce a content file (a video file, a stillimage file, a music file, a document file, an application file, etc.)stored in the storage unit 240 to provide to the user.

FIG. 15 illustrates an embodiment in which the storage unit 240 isprovided in a separate manner from the controller 270, but the scope ofthe present invention is not limited to this. The storage unit 240 maybe included in the controller 270.

The user input interface unit 250 may transfer the user's input signalsto the controller 270 or transfer signals received from the controller270 to the user.

For example, the user input interface unit 250 may receive and processcontrol signals, such as power on/off, channel selection, screen setupand the like, generated from the remote control device 400 or transmitand process control signals generated from the controller 270 to theremote control device 400 according to various communication methods,such as radio frequency (RF) communication, infrared (IR) communicationand the like.

Furthermore, for example, the user input interface unit 250 may transfercontrol signals received from a local key (not shown), such as a powerkey, a channel key, a volume key, a setting key and the like, to thecontroller 270.

Furthermore, for example, the user input interface unit 250 may transfercontrol signals received from a sensing unit (not shown) for sensing theuser's gesture to the controller 270 or transmit signals generated fromthe controller 270 to the sensing unit (not shown). Here, the sensingunit (not shown) may include a touch sensor, a voice sensor, a locationsensor, an operation sensor, and the like.

The controller 270 may inverse-multiplex a stream received from thetuner 210, demodulation unit 220 or external device interface unit 235,and otherwise, process the inverse-multiplexed signals to generate oroutput signals for video or audio output.

The video signal that has been image-processed in the controller 270 maybe input to the display unit 280 and displayed as video corresponding tothe relevant video signal. Furthermore, the video signal that has beenimage-processed in the controller 270 may be input to an external outputdevice through the external device interface unit 235.

The audio signal processed in the controller 270 may be audio-outputtedto the audio output unit 285. Furthermore, the audio signal processed inthe controller 270 may be input to an external output device through theexternal device interface unit 235.

Though not shown in FIG. 15, the controller 270 may include aninverse-multiplexing unit, a video processing unit and the like.

In addition, the controller 270 may control an overall operation withinthe image display device 200. For example, the controller 270 maycontrol the tuner 210 to tune a RF broadcast signal corresponding to theuser's tuned channel or prestored channel.

Furthermore, the controller 270 may control the image display device 200by the user's command received through the user input interface unit 250or internal program. In particular, a network may be connected thereto,thereby allowing the user's desired application or application list tobe downloaded into the image display device 200.

For example, the controller 270 may control the tuner 210 to receive asignal of the tuned channel according to a predetermined channel selectcommand received through the user input interface unit 250. Then, thecontroller 270 processes video, audio or data signals of the tunedchannel. The controller 270 may allow the user's tuned channelinformation or the like to be outputted through the display unit 280 orthe audio output unit 285 along with the processed video or audiosignal.

For another example, the controller 270 may allow video or audio signalsgenerated from an external device, for example, a camera or camcorder,received through the external device interface unit 235, to be outputtedthrough the display unit 280 or the audio output unit 285 according toan external device video play command received through the user inputinterface unit 250.

On the other hand, the controller 270 may control the display unit 280to display an image. For example, the controller 270 may control abroadcast image received through the tuner 210, an external input imagereceived through the external device interface unit 235, an imagereceived through a network interface unit, or an image stored in thestorage unit 240, to be displayed on the display unit 280. Here, theimage displayed on the display unit 280 may be a still or moving image,and otherwise, may be a 2D or 3D image.

On the other hand, when entering an application view item, thecontroller 270 may control an application or application list within theimage display device 200 or an application or application list that canbe downloaded from an external network.

The controller 270 may control an application downloaded from anexternal network to be installed and driven, in addition to various userinterfaces. Furthermore, the controller 270 may control an image relatedto an application being executed to be displayed on the display unit 280by the user's selection.

On the other hand, though not shown in the drawing, it may furtherinclude a channel browsing processing unit for generating a thumbnailimage corresponding to the channel signal or external input signal. Thechannel browsing processing unit may receive a stream signal (TS) outputfrom the demodulation unit 220, a stream signal output from the externaldevice interface unit 235, or the like to extract an image from thereceived stream signal, thereby generating a thumbnail image. Thegenerated thumbnail image may be encoded as it is, to be input to thecontroller 270. Furthermore, the generated thumbnail image may be alsoencoded in a stream type to be input to the controller 270. Thecontroller 270 may display a thumbnail list having a plurality ofthumbnail images on the display unit 280 using an input thumbnail image.On the other hand, thumbnail images within the thumbnail list may besequentially or simultaneously updated. As a result, the user may graspthe content of a plurality of broadcast channels in a convenient manner.

The display unit 280 may convert video, data and OSD signals that areprocessed by the controller 270, video and data signals that arereceived from the external device interface unit 235, or the like, intoR, G, and B signals, respectively, to generate a drive signal.

The display unit 280 may be provided with a PDP, an LCD, an OLED, aflexible display, a 3D display, and the like.

On the other hand, the display unit 280 may be configured with a touchscreen to be used as an input device in addition to an output device.

The audio output unit 285 may receive an audio-processed signal, forexample, a stereo signal, a 3.1-channel signal or a 5.1-channel signalfrom the controller 270 to output it as audio. The audio output unit 285may be implemented by various types of speakers.

On the other hand, a capture unit (not shown) for capturing the user maybe further provided therein. The capture unit (not shown) may beimplemented with one camera but not limited to this, and alsoimplemented with a plurality of cameras. Image information captured bythe capture unit (not shown) may be input to the controller 270.

On the other hand, to detect the user's gesture, the image displaydevice 200 may further include a sensing unit (not shown) having atleast one of a touch sensor, a voice sensor, a location sensor and anoperation sensor as described above. The signal detected by the sensingunit (not shown) may be transferred to the controller 270 through theuser input interface unit 250.

The controller 270 may receive a captured image from the camera unit(not shown) or a detected signal from the sensing unit (not shown)respectively or in a combined manner to detect the user's gesture.

The power supply unit 290 may supply the relevant powers over the entireimage display device 200. In particular, the power supply unit 290 maysupply power to the controller 270 that can be implemented in asystem-on-chip (SOC) form, a display unit 280 for displaying video, andan audio output unit 285 for outputting audio.

For this purpose, the power supply unit 290 may include a converter (notshown) for converting alternating-current power into direct-currentpower. On the other hand, for example, when the display unit 280 isimplemented as a liquid crystal panel having a plurality of backlightlamps, an inverter (not shown) capable of performing a PWM operation maybe further included therein for brightness variation or dimming driving.

The remote control device 400 transmits a user input to the user inputinterface unit 250. For this purpose, the remote control device 400 mayuse various communication techniques such as Bluetooth, Radio Frequency(RF) communication, Infrared (IR) communication, Ultra Wideband (UWB),ZigBee, and the like.

In addition, the remote control device 400 may receive video, audio, ordata signals output from the user input interface unit 250 to display iton the remote control device 400 or output audio or vibration.

The foregoing image display device 200 may be a fixed-type digitalbroadcast receiver capable of receiving at least one of ATSC (8-VSB)broadcast services, DVB-T (COFDM) broadcast services, and ISDB-T(BST-OFDM) broadcast services.

On the other hand, the image display device disclosed herein may be awireless type for transmitting and/or receiving data to and/or from thedisplay unit 280 and audio output unit 285 through wirelesscommunication, as an image display device excluding the display unit 280and audio output unit 285 as illustrated in FIG. 15.

On the other hand, the block diagram of the image display device 200illustrated in FIG. 15 is a block diagram for an embodiment of thepresent disclosure. Each constituent element in the block diagram may beintegrated, added, or deleted according to the specification of anactually implemented image display device 200. In other words, accordingto circumstances, two or more constituent elements may be integratedinto one constituent element, or one constituent element may be dividedinto two or more constituent elements. Furthermore, the function carriedout in each block is provided to describe the embodiment of the presentinvention, and the detailed operation or device will not limit therights scope of the present invention.

On the other hand, the image display device 200, contrary to FIG. 15,may not have the tuner 110 and the demodulation unit 220 as illustratedin FIG. 15, but may receive or play video content through the networkinterface unit 230 or external device interface unit 235.

Moreover, the image display device 200 may include a recording functionunit (not shown) configured to record an external sound to store it asan audio file, and a conversion unit (not shown) configured to convert avoice contained in the sound into a text based on a speech-to-text (STT)conversion. In this case, the controller 270 may detect a core keywordfrom the text, and set the detected core keyword to at least part of afile name for the audio file. Furthermore, the controller 180 mayperform the operation of the first through the seventh embodimentdisclosed herein.

On the other hand, the functions of the mobile terminal 100 according tothe embodiments of the present disclosure will be implemented by arefrigerator which will be described later. Hereinafter, a refrigeratorhaving the operation control function of the mobile terminal 100according to the embodiments of the present disclosure will be describedwith reference to FIG. 16.

FIG. 16 is a block diagram illustrating a display controller and a mainbody controller contained in a refrigerator when an electronic deviceaccording to an embodiment of the present disclosure is therefrigerator.

Referring to FIG. 16, a display unit 310, a communication unit 331, aninput unit 320, a display memory unit 333, a voice recognition unit 335,and a voice output unit 339 are connected to the display controller 330.The display controller 330 controls the operation of the display unit310, communication unit 331, input unit 320, display memory unit 333,voice recognition unit 335, and voice output unit 339.

The communication unit 331 receives image data and DMB data from thebroadcast station or base station to output them to the display unit 310and/or voice output unit 339. Furthermore, the communication unit 331 isconnected to a server or external terminal through a network under thecontrol of the display controller 330 to transmit and receive data. Forexample, the communication unit 331 receives a text query from thedisplay controller 330 to send it to a server or external terminal, andthe server or external terminal transmits a search result for the textquery to the communication unit 331 through a search engine. Thecommunication unit 331 transfers the search result received from theserver or external terminal to the display controller 330, therebyallowing the search result to be output from the display unit 310 and/orvoice output unit 339.

The voice recognition unit 335 may include a voice input unit 336allowing the user to input voice, and a converter 337 configured toconvert the voice input to the audio input unit 336 into a voicefrequency. The converter 337 converts the user's voice frequency into adigital signal and then transfers it to the display controller 330. Thedisplay controller 330 stores the transferred digital signal in thedisplay memory unit 333.

Voice information to be output through the voice output unit 339corresponding to the name of a food stored in the refrigerator is storedin advance in the display memory unit 333. The output voice informationon the name of the food may be stored in advance in the display memoryunit 333 during the fabrication of the refrigerator.

A main body controller 340 is connected to the display controller 330. Amain body memory unit 341 may be connected to the main body controller340. Data for the function of the refrigerator is stored in advance inthe main body memory unit 341.

A recording function unit (not shown) configured to record an externalsound to store it as an audio file, a conversion unit (not shown)configured to convert a voice contained in the sound into a text basedon a speech-to-text (STT) conversion, and the like may be additionallyprovided in the configuration of the refrigerator. The main bodycontroller 340 or display controller 330 detects a core keyword from thetext, and sets the detected core keyword to at least part of a file namefor the audio file. Furthermore, the controller for performing theoperation of the first through the seventh embodiment disclosed hereinmay be additionally provided therein.

According to an embodiment disclosed herein, the foregoing method may beimplemented as codes readable by a processor on a medium written by theprogram. The processor-readable media may include all kinds of recordingdevices in which data readable by a computer system is stored. Examplesof the computer-readable media may include ROM, RAM, CD-ROM, magnetictape, floppy disk, and optical data storage device, and the like, andalso include a device implemented in the form of a carrier wave (forexample, transmission via the Internet).

The configurations and methods according to the above-describedembodiments will not be applicable in a limited way to the foregoingmobile is terminal, image display device, and refrigerator, and all orpart of each embodiment may be selectively combined and configured tomake various modifications thereto.

As described above, the embodiments disclosed herein have been describedwith reference to the accompanying drawings. Here, the terms and wordsused herein and the claims should not be construed by limiting to theirtypical or lexical meaning, but should be construed based on the meaningand concept conforming to the technical concept of the presentinvention.

Accordingly, the configuration illustrated in the embodiments disclosedherein and the drawings is merely the most preferred embodiment of thepresent invention, and is not intended to represent all the technicalconcept of the present invention, and thereby it should be appreciatedthat there may exist various equivalents and modifications forsubstituting those at the time of filing this application.

What is claimed is:
 1. An electronic device, comprising: a storage unitconfigured to store audio data and the recording information of theaudio data; a controller configured to convert an input audio signalinto audio data to store the audio data; a display unit configured todisplay one or more texts based on the execution of a speech-to-text(STT) for the input audio signal; and an input unit configured toreceive a specific pattern input or a selection input for part of thetexts from the user while receiving the audio signal, wherein thecontroller writes a time point at which the specific pattern is receivedor a position of the selected part of the texts on the recordinginformation.
 2. The electronic device of claim 1, wherein the controlleracquires the time point at which the specific pattern is received fromthe recording information of the audio data when there is an accessrequest to the audio data.
 3. The electronic device of claim 2, furthercomprising: a display unit, wherein the controller controls the displayunit to display an indicator indicating a time point at which thespecific pattern is received.
 4. The electronic device of claim 3,wherein the input unit receives the selection of the displayed indicatorfrom the user, and the controller reproduces the audio data from a timepoint corresponding to the indicator selected by the user.
 5. Theelectronic device of claim 1, wherein the audio data and the recordinginformation of the audio data constitute a file.
 6. The electronicdevice of claim 1, wherein the audio data and the recording informationof the audio data constitute a separate file.
 7. The electronic deviceof claim 1, wherein the predetermined pattern input is a touch input toa specific region or a selection input to a specific key button.
 8. Theelectronic device of claim 1, wherein the predetermined pattern is inputcomprises any one of a change of tactile feeling at a specific region, achange of acceleration of the electronic device, and a change of angularspeed of the electronic device.
 9. The electronic device of claim 1,wherein the predetermined pattern input is a specific voice command. 10.The electronic device of claim 1, wherein the predetermined patterninput is a voice with an intensity or volume above a threshold value.11. A method of controlling the operation of an electronic device, themethod comprising: converting an input audio signal into audio data tostore the audio data; displaying one or more texts based on theexecution of a speech-to-text (STT) for the input audio signal;receiving a specific pattern input or an input for part of the textsfrom the user while receiving the audio signal; and writing a time pointat which the specific pattern is received or a position of the selectedpart of the texts on the recording information.
 12. The method of claim11, further comprising: acquiring the time point at which the specificpattern is received from the recording information of the audio datawhen there is an access request to the audio data.
 13. The method ofclaim 12, further comprising: displaying an indicator indicating a timepoint at which the specific pattern is received.
 14. The electronicdevice of claim 13, further comprising: receiving the selection of thedisplayed indicator from the user; and reproducing the audio data from atime point corresponding to the indicator selected by the user.
 15. Themethod of claim 11, wherein the audio data and the recording informationof the audio data constitute a file.
 16. The method of claim 11, whereinthe audio data and the recording information of the audio dataconstitute a separate file.
 17. The method of claim 11, wherein thepredetermined pattern input is a touch input to a specific region or aselection input to a specific key button.
 18. The method of claim 11,wherein the predetermined pattern input comprises any one of a change oftactile feeling at a specific region, a change of acceleration of theelectronic device, and a change of angular speed of the electronicdevice.
 19. The method of claim 11, wherein the predetermined patterninput is a specific voice command.
 20. The method of claim 11, whereinthe predetermined pattern input is a voice an intensity or volume abovea threshold value.